-- Logs begin at Thu 2022-05-12 16:46:54 UTC, end at Thu 2022-05-12 18:03:25 UTC. -- May 12 17:30:10.906790 dv-cinder-840525-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 12 17:30:10.915133 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 12 17:30:10 2022] *** May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: nodename: dv-cinder-840525-2-iscsi May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: machine: x86_64 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: clock source: unix May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: pcre jit disabled May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: detected number of CPU cores: 4 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: current working directory: / May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: detected binary path: /usr/bin/uwsgi-core May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: your processes number limit is 47680 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: your memory page size is 4096 bytes May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: detected max file descriptor number: 2048 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: lock engine: pthread robust mutexes May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: thunder lock: enabled May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 12 17:30:10.919202 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 12 17:30:10.963929 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: Python main interpreter initialized at 0x55bb26fd39c0 May 12 17:30:10.963929 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: python threads support enabled May 12 17:30:10.963929 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: your server socket listen backlog is limited to 100 connections May 12 17:30:10.963929 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: your mercy for graceful operations on workers is 90 seconds May 12 17:30:10.964376 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: mapped 403077 bytes (393 KB) for 2 cores May 12 17:30:10.964376 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: *** Operational MODE: preforking *** May 12 17:30:10.964618 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: *** uWSGI is running in multiple interpreter mode *** May 12 17:30:10.964618 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: spawned uWSGI master process (pid: 80295) May 12 17:30:10.964833 dv-cinder-840525-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 12 17:30:10.975457 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: spawned uWSGI worker 1 (pid: 80296, cores: 1) May 12 17:30:10.975457 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: spawned uWSGI worker 2 (pid: 80297, cores: 1) May 12 17:30:10.975457 dv-cinder-840525-2-iscsi devstack@keystone.service[80295]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 12 17:30:16.891497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 17:30:16.891497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes May 12 17:30:16.892452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 17:30:16.892452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: from cryptography.utils import int_from_bytes May 12 17:30:16.893600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 17:30:16.893600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: from cryptography.utils import int_from_bytes May 12 17:30:16.895230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 12 17:30:16.895230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes May 12 17:30:16.945017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 12 17:30:16.945513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 12 17:30:16.945891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] command line args: [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 12 17:30:16.946374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 12 17:30:16.946747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ================================================================================ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 12 17:30:16.947192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] admin_token = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.947625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_dir = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.947994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.948731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] config_source = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.949108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] debug = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.949616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.950141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.950508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.950944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.951304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 12 17:30:16.951505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 12 17:30:16.951697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] command line args: [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 12 17:30:16.951928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.952015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 12 17:30:16.952177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ================================================================================ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 12 17:30:16.952603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.952982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_config_append = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.953464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.953862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_dir = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.954227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.954646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.954824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.955164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_dir = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.955365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.955556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_source = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.955741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] debug = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.955960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.956081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.956254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.956438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.956703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.956953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.957185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.957363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_config_append = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.957539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.957715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_dir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.957795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.957931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.958135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.958382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.958628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.958872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.959055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.959215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.959310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.959456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.959694 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.959924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.960155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.960264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.960409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.960638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.960880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.961124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.961216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.961393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.961814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.961901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.962052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] publish_errors = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.962283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.962511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.962685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.962928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.963159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.963359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.963452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] standard_threads = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.963608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.963848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.964020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.964211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_journal = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.964387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_json = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.964559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_stderr = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.964731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_syslog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.964958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.965215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.965308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.965446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.965699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.965935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.966180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.966413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.966587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.966681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.966879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.967123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.967366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.967617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.967702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.967849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.968085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.968364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.968610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.968799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.968951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.969138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.969345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.969706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.969920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.970004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.970171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.970425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.970684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.970934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.971111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.971195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.971397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.971626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.971853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.972083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.972375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.972581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.972741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.972878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.973056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.973300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.973549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.973782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.974026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.974254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.974483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.974715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.974943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.975028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.975174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.975387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.975613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.975844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.976041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] publish_errors = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.976125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.976358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.976594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.976821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.977053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.977272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.977364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.977516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.977744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.977977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.978220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.978455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.978539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.978683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.978916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.979142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.979389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.979484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.979628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.979847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.980076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.980359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.980592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.980818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.981002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.981092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.981268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.981533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.981769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.982016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.982189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.982270 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.982473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.982705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.982935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.983178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.983364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] standard_threads = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.983447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.983638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.983877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.984111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.984395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.985120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.985203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.985496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.985730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.985981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.986136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.986346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.986586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.986825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.986910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.987062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.987300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.987547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.987771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_journal = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.987863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.988058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.988380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.988550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.988785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.989014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.989203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_json = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.989412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.989726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_stderr = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.990128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.990469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] use_syslog = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.990563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.990911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.991140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 12 17:30:16.991338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.991762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.991957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.992185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.992576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.992663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.external = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.993104 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.993268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.993679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.993767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.994124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.994375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.994471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.password = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.994672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.token = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.994951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.995192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.995416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.995501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.995696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.995948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.996207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.996442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.996677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.996771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.996918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.997203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.997452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.997689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.997781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.997964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.998201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.998463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.998716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.998903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.998994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.999195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.999478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.999763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:16.999862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.000055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.000866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.001122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.001353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.001585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.001762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.001845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.002055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.002281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.002524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.002756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.002963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.003047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.003224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.003474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.003705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.003955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.004209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.004296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.004443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.004690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.004920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.005145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.005394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.005576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.005667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.005843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.006090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.006336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.006565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.006781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.006865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.007010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.007250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.007491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.007733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.007961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.008046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.008236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.008507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.008826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.008923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.009126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.009360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.009595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.009823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.010055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.010150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.010285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.010523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.010768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.010996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.011227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.011323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.011472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.011704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.011946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.012202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.012434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.012635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.012719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.012880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.013115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.013347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.013585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.013826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.013912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.014065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.014314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.014557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.014786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.015014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.015183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.015300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.015492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.015745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.015985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.016237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.016463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.016701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.016951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.017176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.017418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.017603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.017688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.017886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.018114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.018338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.018567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.018797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.018991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.019090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.019238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.019482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.019723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.019962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.020221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.020321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.020471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.020687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.020929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.021161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.021393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.021567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.021651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.021841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.022081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.022322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.022554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.022731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.022815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.023013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.023251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.023481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.023709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.023930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.024016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.024183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.024430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.024655 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.024887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.025121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.025292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.025376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.025593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.025824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.026068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.026314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.026520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.026615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.026782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.027016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.027251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.027480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.027724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.027811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.027943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.028215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.028465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.028707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.028946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.029133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.029225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.029416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.029659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.029913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.030143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.030322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.030406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.030616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.030858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.031100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.031885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.032034 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.032269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.032517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.032762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.033007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.033208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.033293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.033470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.033717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.033963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.034201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.034394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.034478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.034702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.034930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.035175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.035426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.035610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.035702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.035885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.036117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.036379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.036611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.036820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.036905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.037072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.037303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.037555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.037798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.038032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.038119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.038267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.038507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.038748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.038979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.039213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.039373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.039456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.039669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.039911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.040137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.040412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.040573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.040729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.040958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.041198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.041513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.041600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.041814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.042055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.042286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.042520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.042686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.042770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.043181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.043604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.044037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.044561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.044801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.045232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.045872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.046297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.046717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.047422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.047856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.048303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.048730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.048830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.049061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.049335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.049593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.049806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.049893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.050102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.050362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.050603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.050852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.050939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.051091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.051266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 12 17:30:17.051682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.052108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.052611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.053039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.053458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.053869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.054322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.054742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.055166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.055585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.056008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.056473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.056895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.057325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.057778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.058193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.058612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.059030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.059590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.060014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.060499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.060928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.061365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.061787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.062205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.063133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.063567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.064025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.064468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.064893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.065325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.065760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.066194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.066613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.067033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.067448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.067861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.068308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.068744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.069158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.069578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.070007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.070435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.070853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.071303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.071744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.072191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.072610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.external = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.073023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.073503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.073937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.074351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.password = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.074763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] auth.token = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.075182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.075607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.076037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.076489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.076932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.077363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.077775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.078466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.078898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.079341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.079783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.080236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.080666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.081083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.081532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.081996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.082417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.082666 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.082898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.083121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.083508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.083980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.084436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.084884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.085307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.085751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.086183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.086615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.087044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.087474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.087885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.088349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.088787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.089232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.089649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.090084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.090500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.090927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.091357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.091727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.091970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.092432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.092852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.093267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.093711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.094792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.095296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.095745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.096223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.096653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.097095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.097520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.097956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.098394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.098844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.099303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.099743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.100257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.100693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.101122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.101541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.101982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.102402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.102828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.103256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.103687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.104121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.104592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.105043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.105479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.105903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.106343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.106778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.107213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.107646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.108073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.108518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.108941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.109385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.109923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.110478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.110941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.111378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.111796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.112253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.112708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.113143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.113570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.113993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.114417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.115082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.115520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.115963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.116411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.116848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.117295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.117728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.118164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.118580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.119000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.119421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.119851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.120288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.120743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.121177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.121266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.121446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.121873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.122323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.122771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.123190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.123626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.124050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.124493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.124919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.126044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.126641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.127083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.127512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.127968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.128414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.128868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.129306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.129727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.130154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.130590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.131021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.131451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.131876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.132344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.132786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.133223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.133647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.134101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.134536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.134971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.135404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.135826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.136297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.136722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.137163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.137582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.138014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.138448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.138899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.139334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.139769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.140236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.140701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.141379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.141867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.142308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.142748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.143185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.143622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.144286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.144735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.145176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.145617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.146056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.146494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.146919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.147394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.147835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.148303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.148727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.149149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.149569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.150028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.150549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.150989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.151646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.152080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.152529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.152976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.153418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.153856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.154282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.154729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.155167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.155611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.156045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.157047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.158036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.158469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.158898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.159341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.159776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.160250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.160684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.161126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.161619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.162086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.162533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.163045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.163472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.163899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.164353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.164784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.165223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.165648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.166079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.166516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.166946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.167368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.167823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.168268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.168703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.169119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.169544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.169967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 12 17:30:17.170337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80296) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 12 17:30:17.181193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.197897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.198418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.198855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.207864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.226995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.227444 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.234356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.235228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.277365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.309613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.309905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.310089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.310310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.310527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.310740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.318870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.319332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.333799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.334033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.334249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.334474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.334749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.334967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.335181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.347258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.368628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.368804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.369042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.369129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.369346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.369562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.369775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.369987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.370326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.370759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.371190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.371602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.372013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.386883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.387440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.387894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.387993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.388245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 17:30:17.388534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.388735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 17:30:17.389008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.389250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.389342 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.389518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.389754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.390105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.390523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.390963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.400485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.401630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.401921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.402286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.402569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.404853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.405302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.405727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.406152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.407109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.408341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.408761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80296) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 12 17:30:17.411453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.414738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.415012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.415411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.415690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.415969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.416249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.416515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.416782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.417046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.417314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.417579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.425726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.426178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 17:30:17.426722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.427130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 12 17:30:17.427597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.428054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.428545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.429017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.440776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.441265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.441731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.442238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.445079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.446680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.446955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.451513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.452150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.452445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.452721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.456826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.457320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.457797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.458315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.458804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.459359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.459837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.460346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.460520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.460749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.461186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.461673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.462156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.465311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.465591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.465868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.466141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.466410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.466678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.466944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.467718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.492013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.492313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.493545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.495435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.495938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.497106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.500969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.501168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.502969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.503470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.503949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.506042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.509771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.512032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.512572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.513466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.513757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.514037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.514316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.514592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.514865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.520316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.520828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.521310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.521821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.522293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.522762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.523244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.523708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.534769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.535143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.535409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.535670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.543672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.545371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.545757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.548761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.549039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.549312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.549564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.549842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.560521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.561278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.563232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.563456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.563724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.564008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.564308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.564579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.564879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.573333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.576969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.577249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.584695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.585704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.591771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.595783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.596460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.598063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.599563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.599759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.599871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.600067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.600153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.600381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.600470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.600706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.600948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.609140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.613283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.619080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.619383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.619669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.619948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.624730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.625172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.626177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.626461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.629382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.629683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.629965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.634356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.636070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.636372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.637838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.638120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.638402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.638680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.638955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.639254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.639405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.639552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.639709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.639789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.639973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.640058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.640292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.640390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.640571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.640655 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.641414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.641825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.642095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.654890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.655171 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.655448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.655733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.656012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.656320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.657773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.676396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.679651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.681809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bb26fd39c0 pid: 80297 (default app) May 12 17:30:17.682979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.684596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.685287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130fab6-bcec-494a-b403-8a0c7587e87e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:17.685588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130fab6-bcec-494a-b403-8a0c7587e87e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:17.685868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130fab6-bcec-494a-b403-8a0c7587e87e None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:17.686215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.686496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.686777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.690593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Thu May 12 17:30:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:17.694826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.696395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.699560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.699856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.700140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.700440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.710863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.711173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.711457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.720776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80296) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 12 17:30:17.721064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.721359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.721637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.722349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.722349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.722597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.722783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.723029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.723303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.723572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.723839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.724099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.724398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80296) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 12 17:30:17.765129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bb26fd39c0 pid: 80296 (default app) May 12 17:30:25.830002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93069d91-cbdd-479f-b783-a329067b9027 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:25.830569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93069d91-cbdd-479f-b783-a329067b9027 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:25.830569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93069d91-cbdd-479f-b783-a329067b9027 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:25.831880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 2/2] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:25.839490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:25.839939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:25.840111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:25.872209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG oslo_db.sqlalchemy.engines [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80296) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 12 17:30:26.060837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG passlib.handlers.bcrypt [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80296) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 12 17:30:26.061253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG passlib.handlers.bcrypt [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80296) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 12 17:30:26.117550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:26.119159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:26.242281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1/3] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 12 17:30:26.373065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93069d91-cbdd-479f-b783-a329067b9027 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:26.406974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG oslo_db.sqlalchemy.engines [None req-93069d91-cbdd-479f-b783-a329067b9027 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80297) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 12 17:30:26.532669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93069d91-cbdd-479f-b783-a329067b9027 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:26.536828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:26.537480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:26.537757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:26.538029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:26.545981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:26.598127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 12 17:30:26.598467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4c0cd-05bb-44fa-a399-03f1d1f3e2a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:26.608261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 3/4] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 12 17:30:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:30:26.614708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:26.630249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80cfbd-8ec8-43fb-9592-45284e711397 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:26.634858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:26.636027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:26.636350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:26.636624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] PATH_INFO: `/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:26.638205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] RBAC: Authorizing `identity:get_project(project_id=ccf7092b8d0b4d6a9522dc9ba6d91833)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:26.689303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 12 17:30:26.689633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d63fd-8a00-44fe-b3c4-6f7505fc9962 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:26.694768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 2/5] 10.222.0.73 () {56 vars in 1207 bytes} [Thu May 12 17:30:26 2022] GET /identity/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:28.918845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f88dd-9fc1-481e-a244-a4583fc43d93 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:28.919355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f88dd-9fc1-481e-a244-a4583fc43d93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:28.919438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f88dd-9fc1-481e-a244-a4583fc43d93 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:28.924236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 4/6] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:28.929676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:28.930063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:28.930232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:28.951108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:28.952641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:28.960918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 3/7] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 12 17:30:28.982828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f88dd-9fc1-481e-a244-a4583fc43d93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:28.995498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f88dd-9fc1-481e-a244-a4583fc43d93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:29.000877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf16fe-f159-49f3-a8eb-0a1f2447251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:29.001434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf16fe-f159-49f3-a8eb-0a1f2447251d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:29.001717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf16fe-f159-49f3-a8eb-0a1f2447251d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:29.001992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf16fe-f159-49f3-a8eb-0a1f2447251d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:29.009553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf16fe-f159-49f3-a8eb-0a1f2447251d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:29.011312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf16fe-f159-49f3-a8eb-0a1f2447251d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:29.019890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 5/8] 10.222.0.73 () {58 vars in 1324 bytes} [Thu May 12 17:30:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:30:29.025392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:29.035961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a83a4-3a73-4d46-9804-a2342666b833 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:29.056896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68421ced-ae35-4eef-92ec-1a40e3eeb473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:29.057780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68421ced-ae35-4eef-92ec-1a40e3eeb473 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:29.058062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68421ced-ae35-4eef-92ec-1a40e3eeb473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:29.058345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68421ced-ae35-4eef-92ec-1a40e3eeb473 admin admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:29.059778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68421ced-ae35-4eef-92ec-1a40e3eeb473 admin admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:29.063762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68421ced-ae35-4eef-92ec-1a40e3eeb473 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:29.065410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 4/9] 10.222.0.73 () {56 vars in 1198 bytes} [Thu May 12 17:30:29 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:31.335307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ff850-b81d-4a90-ba53-9e819eeedf5e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.335847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ff850-b81d-4a90-ba53-9e819eeedf5e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.335847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ff850-b81d-4a90-ba53-9e819eeedf5e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.337219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 6/10] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:31.342523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.342918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.343192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.376426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8899eed-a10a-43f0-ab50-c800e4d80552 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.376706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8899eed-a10a-43f0-ab50-c800e4d80552 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.376979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8899eed-a10a-43f0-ab50-c800e4d80552 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.378349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 7/11] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:31.384460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.384748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.385027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.428147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80297) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 12 17:30:31.428598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80297) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 12 17:30:31.473733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:31.474928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:31.475295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:31.476888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:31.553063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 5/12] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:31.574474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 8/13] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:30:31.576904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:31.597101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:31.675236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf45ac-7da1-4c55-98f4-c6e8dae8aaea None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:31.695584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e38619-8e7a-401a-b370-ee005f28c1da None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:31.707733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:31.708595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.708885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.709164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.720736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:31.729951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:31.730851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.731138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.731413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:31.731861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:31.732773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:30:31.733573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 6/14] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:30:31.739643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:31.795539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:31.797143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:31.834715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590c7da-64b5-4c62-ad37-1611261b3ed3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:31.867448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:31.868054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:31.868382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:31.868662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:31.869414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:31.869721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:31.871107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:31.882521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 7/15] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:31.887921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:31.888826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 9/16] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 17:30:31.894740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:31.982596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e76fb-9a95-4c29-8062-146ab61e9bb2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:31.988690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e11e1-cc9c-44f3-a1db-e004f5469279 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:32.014046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:32.014906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:32.015187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:32.015950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:32.017163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:32.018605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:32.019638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:32.020211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 8/17] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:31 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:32.020452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:32.020732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:32.020999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:32.025411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:32.127525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bfa35-fa28-4257-b0d7-e33ffbc9327b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:32.146706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:32.148183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:32.160322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:32.161118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:32.161398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:32.161671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:32.162853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:32.164320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:32.165918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 9/18] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:30:32.169093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:32.170035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:32.171026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:30:32.171026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:30:32.171026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:30:32.171026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:32.171026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 10/19] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:30:32.172677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:32.175991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:32.266662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9dd43-06ef-4c72-aec6-c28d18df9029 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:32.272233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429435de-1659-4297-902e-cf5581a073f3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:32.297864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:32.298517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:32.298802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:32.299075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:32.299902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.domain.id=default, user.id=254e338acf294dff9fffc87141257a5d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:32.300242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:32.301684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6dd5a-6aec-4452-8b15-8ae53b222364 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:32.303505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:32.304180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:32.304475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:32.304756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:32.305486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:32.305796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:32.307163 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fdc55-1f01-48c4-84c8-16ce506ec907 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:32.308253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 10/20] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 12 17:30:32 2022] GET /identity/v3/role_assignments?user.id=254e338acf294dff9fffc87141257a5d&scope.domain.id=default&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 233 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:32.326475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 11/21] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:35.632620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024511a7-906a-4da4-a5a8-9a5a9583958a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:35.632620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024511a7-906a-4da4-a5a8-9a5a9583958a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:35.632620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024511a7-906a-4da4-a5a8-9a5a9583958a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:35.633572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 11/22] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:35.638953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:35.639371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:35.639510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:35.669219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:35.670424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:35.712220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526bc44-73b5-4272-9124-760f863e4d79 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:35.712220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526bc44-73b5-4272-9124-760f863e4d79 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:35.712220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526bc44-73b5-4272-9124-760f863e4d79 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:35.714699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 12/23] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:35.721055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794e65-6528-48b3-833c-e4011dac647b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:35.721055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794e65-6528-48b3-833c-e4011dac647b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:35.721055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794e65-6528-48b3-833c-e4011dac647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:35.777740 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-30794e65-6528-48b3-833c-e4011dac647b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:35.778965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-30794e65-6528-48b3-833c-e4011dac647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:35.852677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 12/24] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:35.892428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:35.995766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 13/25] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:36.056666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30794e65-6528-48b3-833c-e4011dac647b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:36.200844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edb146-2f55-4db7-bbf3-3284bb807c2b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:36.243790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97244809-3250-431e-85bf-d9aac2d6bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:36.244891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97244809-3250-431e-85bf-d9aac2d6bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.245312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97244809-3250-431e-85bf-d9aac2d6bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.245724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97244809-3250-431e-85bf-d9aac2d6bcd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.259794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97244809-3250-431e-85bf-d9aac2d6bcd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:36.266143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30794e65-6528-48b3-833c-e4011dac647b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-97244809-3250-431e-85bf-d9aac2d6bcd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:36.271367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:36.272364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:36.273120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:36.274122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 13/26] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:36.275284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64c379-614c-4761-93c0-5d69e9323b0b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.275653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64c379-614c-4761-93c0-5d69e9323b0b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.276001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64c379-614c-4761-93c0-5d69e9323b0b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.279200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 14/27] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:36.281653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75e5c3-05ef-4e46-868f-531e04f58801 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.282179 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75e5c3-05ef-4e46-868f-531e04f58801 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.282650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75e5c3-05ef-4e46-868f-531e04f58801 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.284557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 15/28] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:36.286035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a228f-577f-48fc-9eb1-98aa63fbda22 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.286521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a228f-577f-48fc-9eb1-98aa63fbda22 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.287009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a228f-577f-48fc-9eb1-98aa63fbda22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.313230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a9a228f-577f-48fc-9eb1-98aa63fbda22 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:36.315304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a9a228f-577f-48fc-9eb1-98aa63fbda22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:36.318598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:36.319638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.320127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.320622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.333909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:36.335673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:30:36.364393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:36.365603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:36.366762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:30:36.366762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:30:36.366762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:30:36.366762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:30:36.368750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 14/29] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:36.369713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:36.401075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 16/30] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:36.403079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.404326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.404835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.422996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:36.424529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:36.509755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b059dfc-f620-47cf-a4d8-f277979b45e7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:36.511617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 17/31] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:36.512827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:36.558032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:36.558724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.559010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.559289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.560508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:36.560989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:36.563113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:36.575013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 15/32] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:36.575872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:36.636813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd81c9-e717-4a5d-99fb-5fb43c83cddf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:36.681448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:36.682857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.683279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.683754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.684796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:36.685535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:36.687026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6802336-b8c8-4dfd-a1bb-94f94a3bae9b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:36.695516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1e07-226c-4467-b697-0a23f717448c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:36.708149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 18/33] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:36.709907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc7f5-292e-4ec1-9702-cfb519ecadde None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.710334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc7f5-292e-4ec1-9702-cfb519ecadde None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.710767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc7f5-292e-4ec1-9702-cfb519ecadde None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.716918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 19/34] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:36.717757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc7f5-292e-4ec1-9702-cfb519ecadde None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:36.732577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:36.733252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.733533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.733808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.735299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:36.736723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:36.833351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc7f5-292e-4ec1-9702-cfb519ecadde None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:36.864371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 16/35] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 12 17:30:36 2022] POST /identity/v3/roles => generated 223 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:36.865867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:36.868482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:36.869370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:36.869803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:36.870102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:36.870873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:36.872470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:36.986820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a8b99-60de-48df-a5b7-6abe800c6917 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.000333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 20/36] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 12 17:30:36 2022] POST /identity/v3/roles => generated 219 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:37.001577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:37.047807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4758-16f4-4f30-a40b-06ec6506cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.049188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4758-16f4-4f30-a40b-06ec6506cae1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.049691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4758-16f4-4f30-a40b-06ec6506cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.050189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4758-16f4-4f30-a40b-06ec6506cae1 None admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.052028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4758-16f4-4f30-a40b-06ec6506cae1 None admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.054294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4758-16f4-4f30-a40b-06ec6506cae1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.061600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 17/37] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:36 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:37.064026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0cde0-493c-4446-b1b4-be8d4b7c7f2f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.064559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0cde0-493c-4446-b1b4-be8d4b7c7f2f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.065297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0cde0-493c-4446-b1b4-be8d4b7c7f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.087294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-35f0cde0-493c-4446-b1b4-be8d4b7c7f2f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:37.089253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-35f0cde0-493c-4446-b1b4-be8d4b7c7f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:37.131654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30ddbc-a5f4-42ea-b1ee-ffa5f760e266 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.189335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643484d-3bcb-49db-8d09-a991ef16061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.190011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643484d-3bcb-49db-8d09-a991ef16061c None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.190298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643484d-3bcb-49db-8d09-a991ef16061c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.190577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643484d-3bcb-49db-8d09-a991ef16061c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.191578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643484d-3bcb-49db-8d09-a991ef16061c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.195411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643484d-3bcb-49db-8d09-a991ef16061c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.212468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 18/38] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:37.214456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed8306-bd46-483f-b0c9-21be0212671a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.216605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed8306-bd46-483f-b0c9-21be0212671a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.217286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed8306-bd46-483f-b0c9-21be0212671a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.224299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 19/39] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:37.225752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed8306-bd46-483f-b0c9-21be0212671a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:37.241861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 21/40] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:30:37 2022] POST /identity/v3/projects => generated 301 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:37.243471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.243763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.244041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.267137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:37.269116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:37.366165 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 22/41] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:37.367011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:37.419284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed8306-bd46-483f-b0c9-21be0212671a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.464037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.464858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.465139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.465407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.467131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.469326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.471801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 20/42] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:37.473224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:37.476243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf054e-8591-47c9-a4d2-9ed357f57947 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.513625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.514301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.514581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.514857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.515637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.517338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.530650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 23/43] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 12 17:30:37 2022] POST /identity/v3/roles => generated 225 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:37.532179 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:37.605469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44e3e8-6d15-4922-abd7-ea8b9d855116 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.652037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.652816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.653098 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.653372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.654626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.656309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.657248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66865-d886-4d52-ba5e-2d22b6459567 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.662792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 21/44] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:37.663925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:37.696247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502877b7-12aa-4d3b-84a0-f19c8dd26937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.697151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502877b7-12aa-4d3b-84a0-f19c8dd26937 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.697436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502877b7-12aa-4d3b-84a0-f19c8dd26937 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.697721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502877b7-12aa-4d3b-84a0-f19c8dd26937 None admin] PATH_INFO: `/v3/domains/default/users/254e338acf294dff9fffc87141257a5d/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.700324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502877b7-12aa-4d3b-84a0-f19c8dd26937 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=254e338acf294dff9fffc87141257a5d, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.701845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502877b7-12aa-4d3b-84a0-f19c8dd26937 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.721052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 24/45] 10.222.0.73 () {58 vars in 1361 bytes} [Thu May 12 17:30:37 2022] PUT /identity/v3/domains/default/users/254e338acf294dff9fffc87141257a5d/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:30:37.788726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9c32f-e904-4d08-9611-1d5d39ac0d0a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:37.837731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece0f81-c7c6-4258-8c63-f9eb9213e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:37.838406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0f81-c7c6-4258-8c63-f9eb9213e790 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:37.838687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0f81-c7c6-4258-8c63-f9eb9213e790 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:37.838965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0f81-c7c6-4258-8c63-f9eb9213e790 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:37.839689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece0f81-c7c6-4258-8c63-f9eb9213e790 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:37.841223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece0f81-c7c6-4258-8c63-f9eb9213e790 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:37.872294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 22/46] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:30:37 2022] POST /identity/v3/projects => generated 312 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:40.018717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13434-9339-4413-968f-d95dd91c5512 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.019228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13434-9339-4413-968f-d95dd91c5512 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.019302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13434-9339-4413-968f-d95dd91c5512 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.020802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 25/47] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:40.028788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6921c-c624-4777-ba6b-d5aad84f82c9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.029293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6921c-c624-4777-ba6b-d5aad84f82c9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.029369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6921c-c624-4777-ba6b-d5aad84f82c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.047725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0ef6921c-c624-4777-ba6b-d5aad84f82c9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:40.048970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ef6921c-c624-4777-ba6b-d5aad84f82c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:40.119031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 23/48] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:40.143762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f13434-9339-4413-968f-d95dd91c5512 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.190751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1444903-2078-4aa2-ac9d-eb572eac38b4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.191073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1444903-2078-4aa2-ac9d-eb572eac38b4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.191354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1444903-2078-4aa2-ac9d-eb572eac38b4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.192753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 24/49] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:40.198004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.198296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.198807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.215848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:40.217182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:40.264820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f13434-9339-4413-968f-d95dd91c5512 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.290503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 25/50] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:30:40.303802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.304641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.304920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.305187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.313885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.316326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:40.324078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:40.324946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:40.325674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 26/51] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:30:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:40.332613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.417586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24b3f6-51d2-4a38-b921-58553eb43e33 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.438778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb38db-0398-40ea-9bcb-d4fd146914e3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.454503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.455300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.455578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.455850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.457080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:40.458527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:40.460125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 26/52] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:40.465759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.473933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.474614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.474900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.475179 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.475934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:40.476273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:40.477664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:40.490875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 27/53] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:40.496563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.581574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d68141-465c-4163-b42d-caccb5e0ecdf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.607419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9e3e5-fb10-419e-b0eb-7dac34ad0d20 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.620435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.621103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.621545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.621821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.622679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:40.624275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:40.643602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.644422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.644676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.644951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.646184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:40.647643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:40.649375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 28/54] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:40 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:40.654252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 27/55] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 12 17:30:40 2022] POST /identity/v3/projects => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:40.654972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.768055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb71fc1-0fb1-4ade-b455-753d764eb833 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.807843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8af3ba-ad4c-49e7-b5c2-7ada9d389d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.809277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8af3ba-ad4c-49e7-b5c2-7ada9d389d5d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.809571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8af3ba-ad4c-49e7-b5c2-7ada9d389d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.809879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8af3ba-ad4c-49e7-b5c2-7ada9d389d5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.811795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8af3ba-ad4c-49e7-b5c2-7ada9d389d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:40.814313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8af3ba-ad4c-49e7-b5c2-7ada9d389d5d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:40.820216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 29/56] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:40.825407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:40.941702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66ca84-ab8b-4026-b19f-d7dc6d8cb4bb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:40.976426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:40.977094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:40.977383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:40.977668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:40.978507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.domain.id=default, user.id=254e338acf294dff9fffc87141257a5d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:40.978822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:40.980278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200ac6-af03-41a5-9d58-84ec606ad165 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:40.986932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 28/57] 10.222.0.73 () {56 vars in 1322 bytes} [Thu May 12 17:30:40 2022] GET /identity/v3/role_assignments?user.id=254e338acf294dff9fffc87141257a5d&scope.domain.id=default&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:42.944155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afb5b8-9f7a-4d90-8702-652c2de53a8c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:42.944708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afb5b8-9f7a-4d90-8702-652c2de53a8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:42.944708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afb5b8-9f7a-4d90-8702-652c2de53a8c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:42.946090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 30/58] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:42.951225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:42.951638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:42.951792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:42.969499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:42.970679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:43.033021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 29/59] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:30:43.057176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afb5b8-9f7a-4d90-8702-652c2de53a8c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:43.151303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afb5b8-9f7a-4d90-8702-652c2de53a8c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:43.182467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fda33-63b1-49d5-9c94-d18dd473f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:43.183273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fda33-63b1-49d5-9c94-d18dd473f45a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:43.183555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fda33-63b1-49d5-9c94-d18dd473f45a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:43.183830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fda33-63b1-49d5-9c94-d18dd473f45a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:43.185035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fda33-63b1-49d5-9c94-d18dd473f45a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:43.186486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fda33-63b1-49d5-9c94-d18dd473f45a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:43.188525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 31/60] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:43.193905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:43.286264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a0b52-025b-49eb-b62f-138ae6def5d7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:43.317642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeeaeb-7274-464e-a49d-f3bd5fd1d949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:43.318290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeeaeb-7274-464e-a49d-f3bd5fd1d949 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:43.318575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeeaeb-7274-464e-a49d-f3bd5fd1d949 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:43.318852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeeaeb-7274-464e-a49d-f3bd5fd1d949 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:43.319551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeeaeb-7274-464e-a49d-f3bd5fd1d949 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:43.321018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeeaeb-7274-464e-a49d-f3bd5fd1d949 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:43.345994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 30/61] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:30:43 2022] POST /identity/v3/users => generated 275 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:46.871835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e8d5b-2658-45a8-8c9e-9a4ec2b5f16f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:46.872878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e8d5b-2658-45a8-8c9e-9a4ec2b5f16f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:46.873366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e8d5b-2658-45a8-8c9e-9a4ec2b5f16f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:46.875123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 32/62] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:46.881069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:46.882044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:46.882543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:46.909133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:46.910868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:46.955432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8457c3-c51a-49f3-bcb5-88284151deb0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:46.956065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8457c3-c51a-49f3-bcb5-88284151deb0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:46.956575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8457c3-c51a-49f3-bcb5-88284151deb0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:46.958200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 33/63] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:46.965091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b2f7d-bd38-46f3-b782-f630337bf07f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:46.966327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b2f7d-bd38-46f3-b782-f630337bf07f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:46.966791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b2f7d-bd38-46f3-b782-f630337bf07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.000148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 31/64] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:47.001351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4a5b2f7d-bd38-46f3-b782-f630337bf07f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:47.002712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a5b2f7d-bd38-46f3-b782-f630337bf07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:47.039728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.085485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 34/65] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:30:47.111652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4ce1-ab18-4113-88eb-a3627b87297d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.112282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4ce1-ab18-4113-88eb-a3627b87297d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.112760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4ce1-ab18-4113-88eb-a3627b87297d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.114385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 35/66] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:47.115553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4ce1-ab18-4113-88eb-a3627b87297d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.159891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9635e-a44f-4016-b5c9-d9bc58f3f9f5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.196715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eed78-dd61-497d-a466-a9957396b0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.197878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eed78-dd61-497d-a466-a9957396b0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.198761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eed78-dd61-497d-a466-a9957396b0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.199237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eed78-dd61-497d-a466-a9957396b0f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.212512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7eed78-dd61-497d-a466-a9957396b0f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fa7eed78-dd61-497d-a466-a9957396b0f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:47.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:47.224517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:47.225266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:30:47.225850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4ce1-ab18-4113-88eb-a3627b87297d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.228397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 32/67] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:30:47.229820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45048cb-9cf2-4db2-9c7c-8b1af48fdfa6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.230251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45048cb-9cf2-4db2-9c7c-8b1af48fdfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.230692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45048cb-9cf2-4db2-9c7c-8b1af48fdfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.251876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a45048cb-9cf2-4db2-9c7c-8b1af48fdfa6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:47.253701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a45048cb-9cf2-4db2-9c7c-8b1af48fdfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:47.262893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.263684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.263966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.264266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.276199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:47.284335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:47.285166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:47.285903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 36/68] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:47.286516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.336205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 33/69] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:47.336205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c69bc-2823-415a-bc57-f32d54524ddf None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.336205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c69bc-2823-415a-bc57-f32d54524ddf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.336205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c69bc-2823-415a-bc57-f32d54524ddf None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.336205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 34/70] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:47.336205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c69bc-2823-415a-bc57-f32d54524ddf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.393987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3aa50-243d-4a72-8086-53643118dadf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.425817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.426406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.426691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.426965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.427707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:47.428011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:47.429424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f53d8-ba2d-47d9-b1e0-569c1f57605b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:47.429671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c69bc-2823-415a-bc57-f32d54524ddf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.437134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 37/71] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:47.438309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.438603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.438880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.454712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:47.455887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:47.462516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.463184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.463472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.463746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.464522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:47.464829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:47.466201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826221-a03d-40ce-a1f7-04b8ec1e5029 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:47.474626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 35/72] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:47.475819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cefa79-d04d-4667-b509-d57c6b94102b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.476109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cefa79-d04d-4667-b509-d57c6b94102b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.476420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cefa79-d04d-4667-b509-d57c6b94102b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.477825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 36/73] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:47.480200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cefa79-d04d-4667-b509-d57c6b94102b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.519583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 38/74] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:47.528357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.584670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cefa79-d04d-4667-b509-d57c6b94102b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.611727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71602858-0286-4e17-8b7f-dcdcdc464050 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.619389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.620225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.620522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.620804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.630984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:47.638914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:47.639744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:30:47.641300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 37/75] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:47.642072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.648789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7fb56-20b6-46b8-9a83-000f687dd1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.649669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7fb56-20b6-46b8-9a83-000f687dd1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.649952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7fb56-20b6-46b8-9a83-000f687dd1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.650226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7fb56-20b6-46b8-9a83-000f687dd1e6 None admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.651425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7fb56-20b6-46b8-9a83-000f687dd1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:47.652917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7fb56-20b6-46b8-9a83-000f687dd1e6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:47.654537 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 39/76] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:47.655722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.656182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.656625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.673806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:47.674969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:47.736216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 40/77] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:47.737012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.739464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7c05d-388c-4eb9-8fbb-ce7fa4d67fe7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.772116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.773094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.773379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.773654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.788063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:47.789548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:47.791383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 38/78] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:47.791939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.843971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca6dee-9c99-4099-ac10-3b2533ad4ac8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.876930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.877714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.877996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.878268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.886756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bcfaa-c46b-4373-a63d-bf273a2901f4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:47.887832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:47.895479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:47.896450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:47.897192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 41/79] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:47.898247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.919389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:47.920064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:47.920380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:47.920669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:47.922039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:47.922351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:47.923724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:47.935839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 39/80] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:47.935839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:47.991680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4980a-120a-425a-856b-7e44035d5e6d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.022440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.023075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.023356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.023630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.023817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2e5e8-b93f-46d6-9368-82927f05fa50 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.056096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.056896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.057183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.057462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.058624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.060060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.061680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 40/81] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.062472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.085715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.087159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.157433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a945db-8ea0-4fb2-9fa5-86ddfd4df5bf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.177448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 42/82] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:30:48.178041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.193831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.194489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.194775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.195054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.261583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.263799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.269430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba2d45-6cb1-4b6b-8c9a-a110368c4a7b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.306279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.307173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.307596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.308002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.309153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:48.309578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.311594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.319382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 43/83] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.320193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.358474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 41/84] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:30:48.359306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.415006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8b6e-dff4-4e3b-b683-67ff30bfd949 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.445548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.446413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.446694 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.446969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.448191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.449628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.449808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449924-8842-4d09-a6d9-45d5c2225373 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.451416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 44/85] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.452076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.483575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.484309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.484597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.484871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.485575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.487011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.506448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 42/86] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:30:48 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:48.507246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.548727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350403e-c085-475a-b6d6-b2643f61606e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.595721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.596639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.596922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.597194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.598598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.600523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.607524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 45/87] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.607524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.614186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6ea8-e7ee-4e9e-be4e-2b74d444179b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.651030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.651929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.652224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.652506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.653927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.655583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.660533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 43/88] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.661780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.728849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a01a5-7923-4488-9c42-a8a36ac81307 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.765850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114c5d4-fe25-403e-ae9f-dc6adafa4161 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.767938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.768833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.769125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.769395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.770589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.772046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.773941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 46/89] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.774628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:48.801476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.802071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.802361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.802643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.875178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.878080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.886926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2332e5e-7a7c-4c9b-a364-d5553332e577 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:48.923798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:48.924856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:48.925343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:48.925842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:48.926948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=254e338acf294dff9fffc87141257a5d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:48.927461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:48.929198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:48.938201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 47/90] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/role_assignments?user.id=254e338acf294dff9fffc87141257a5d&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:48.939305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:49.041223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 44/91] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:30:49.042759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:49.123746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5bfe0-dbba-4efe-a412-8dab09224dd1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:49.163886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:49.164397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:49.164686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:49.164959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:49.229995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a2eeb-9ccd-444d-be47-ef15e939d77e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:49.275280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:49.277085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:49.277797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:49.278466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:49.278753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:49.279030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:49.279953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=dbe1e8845ac542ff9fb11a8bbaa70f79) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:49.280198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:49.282644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:49.302289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 45/92] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:49 2022] GET /identity/v3/role_assignments?user.id=dbe1e8845ac542ff9fb11a8bbaa70f79&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:49.303140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:49.464608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 48/93] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:30:49.477023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:49.554453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b77aa-9316-46f0-8636-6a02f0b32b10 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:49.625692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:49.626587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:49.626872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:49.627148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:49.628392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:49.629847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:49.631548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 46/94] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:49 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:49.641146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:49.687447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a013de4-34c8-407d-b68e-515025367ad4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:49.726940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:49.727722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:49.728003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:49.728303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] PATH_INFO: `/v3/projects/46218928188b428a8d7b40eed5bf09aa` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:49.740187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] RBAC: Authorizing `identity:get_project(project_id=46218928188b428a8d7b40eed5bf09aa)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:49.740187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:49.740187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 49/95] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:49 2022] GET /identity/v3/projects/46218928188b428a8d7b40eed5bf09aa => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:49.742545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:49.802029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d3b9e-bbd8-4fd2-937d-e66031c83298 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:49.866664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:49.867334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:49.867622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:49.867896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:49.868919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] RBAC: Adding query filter params (role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=dbe1e8845ac542ff9fb11a8bbaa70f79) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:49.869233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:49.870837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2beed-f722-4a49-84cd-1f4da4877607 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:49.886297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 47/96] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:49 2022] GET /identity/v3/role_assignments?user.id=dbe1e8845ac542ff9fb11a8bbaa70f79&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:49.930451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04630e-3a03-4d31-90e0-4ded0fbaca94 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:49.967602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:49.968379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:49.968662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:49.968940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:49.969908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=46218928188b428a8d7b40eed5bf09aa, user.id=dbe1e8845ac542ff9fb11a8bbaa70f79) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:49.970218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:49.971676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c8f54-3e77-430e-920c-53617db4eb02 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:49.985876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 50/97] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:49 2022] GET /identity/v3/role_assignments?user.id=dbe1e8845ac542ff9fb11a8bbaa70f79&scope.project.id=46218928188b428a8d7b40eed5bf09aa&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:51.823281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f752c9-3a7f-45d3-b215-3166bcf5b23c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:51.823857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f752c9-3a7f-45d3-b215-3166bcf5b23c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:51.823857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f752c9-3a7f-45d3-b215-3166bcf5b23c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:51.825583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 48/98] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:51.842143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:51.842653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:51.842731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:51.860021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1918ce4-907b-43ff-bc7a-a7cce383af27 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:51.860337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1918ce4-907b-43ff-bc7a-a7cce383af27 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:51.860620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1918ce4-907b-43ff-bc7a-a7cce383af27 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:51.861974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 49/99] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:51.867334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:51.867566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:51.867852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:51.873274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:51.874993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:51.900177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:51.901385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:52.026182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 50/100] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:30:52.035128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 51/101] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:52.066959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.072065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.213675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336fb69-e23b-4c9e-9bc7-8705400eb51f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.250546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d2f8d-2dd6-434d-bbd4-169d79b5b2da None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.265860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed61875-72b1-40a4-a917-60f2236d1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.266977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed61875-72b1-40a4-a917-60f2236d1e97 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.267463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed61875-72b1-40a4-a917-60f2236d1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.267922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed61875-72b1-40a4-a917-60f2236d1e97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.278859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed61875-72b1-40a4-a917-60f2236d1e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6ed61875-72b1-40a4-a917-60f2236d1e97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:52.287169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:52.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:30:52.288886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:30:52.291364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 51/102] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:52.292803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31154c9-366b-42e8-b123-261b27ea57da None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.293280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31154c9-366b-42e8-b123-261b27ea57da None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.293739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31154c9-366b-42e8-b123-261b27ea57da None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.295185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 52/103] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:52.296535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49c877-adbe-4d26-add4-9c4766cb8c44 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.297029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49c877-adbe-4d26-add4-9c4766cb8c44 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.297310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49c877-adbe-4d26-add4-9c4766cb8c44 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.298535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 53/104] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:52.299234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49c877-adbe-4d26-add4-9c4766cb8c44 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.308981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16ce85-7997-4370-ad5a-8b90006f9511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.310065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16ce85-7997-4370-ad5a-8b90006f9511 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.310529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16ce85-7997-4370-ad5a-8b90006f9511 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.310973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16ce85-7997-4370-ad5a-8b90006f9511 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.321903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16ce85-7997-4370-ad5a-8b90006f9511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4e16ce85-7997-4370-ad5a-8b90006f9511 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:52.330291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:52.331123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:52.331883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:52.334568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 52/105] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:30:52.336024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657447-b1c1-49a9-9b56-482ffea4b948 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.336509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657447-b1c1-49a9-9b56-482ffea4b948 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.337028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657447-b1c1-49a9-9b56-482ffea4b948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.355659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b9657447-b1c1-49a9-9b56-482ffea4b948 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:52.356883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9657447-b1c1-49a9-9b56-482ffea4b948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:52.418100 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49c877-adbe-4d26-add4-9c4766cb8c44 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.437884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 53/106] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:52.439182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.439966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.440280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.453694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.454461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.454852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.455194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.456051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:52.456459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:52.457939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:52.459035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:52.460680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:52.470064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 54/107] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:52.470847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.545592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 54/108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:52.548693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.592201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940bf5f-9d00-4af0-bd30-ba22c33c58f6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.633927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.634943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.636097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.636620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.637567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:52.638139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:52.639848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:52.649127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 55/109] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:52.650172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.676230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c316a-d2d2-44c3-ae2f-9e1ee9003255 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.720254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.721237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.721527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.721819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] PATH_INFO: `/v3/users/29d940305d7e4597aed3653717311f34` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.742142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] RBAC: Authorizing `identity:get_user(user_id=29d940305d7e4597aed3653717311f34)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:52.743800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:52.746401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 55/110] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/users/29d940305d7e4597aed3653717311f34 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:52.747175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.757150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f36dd-df5c-42b1-b63e-6ab4f7246002 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.792589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.793868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.794365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.794913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.797956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:52.800681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:52.803113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 56/111] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:52.804239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.855285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07c6b-9a6a-4e66-bfca-83dee2556d52 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.890449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.891491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.891777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.892047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.902610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:52.910581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:52.911443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:52.912226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:52.916432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 56/112] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:52.917326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:52.920229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ded3d-ad9b-49d8-bb9b-43cef718960d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:52.956642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:52.957926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:52.958409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:52.958892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:52.960374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:52.962025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:52.964261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 57/113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:52.965217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.018733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1674eed-4b13-44ea-98e5-1e3d03cad116 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.052801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.053460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.053746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.054025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.063951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3032a3-2a95-4d0e-8f18-d611a753c43c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.096025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.096724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.097009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.097290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.097996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.099420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.123940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 58/114] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:30:52 2022] POST /identity/v3/projects => generated 302 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:53.125339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.136250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.136250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.239170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301e77-0150-496c-a8d4-571ea410f337 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.263857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 57/115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:30:53.266073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.277722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.278007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.278423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.278770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.279597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:53.280239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.282148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea66c8-1b05-4680-83cd-81c3b881de19 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.291723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 59/116] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:53.292785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb88e2-ba2e-4436-a5ef-3742b42f6634 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.293143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb88e2-ba2e-4436-a5ef-3742b42f6634 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.293498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb88e2-ba2e-4436-a5ef-3742b42f6634 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.299700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 60/117] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:53.299700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb88e2-ba2e-4436-a5ef-3742b42f6634 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.382680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46c52c-5d0c-495a-89ae-fd723557154c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.401920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb88e2-ba2e-4436-a5ef-3742b42f6634 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.418519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.419378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.419670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.419948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.421303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.422793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.424693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 58/118] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:53.425293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.443699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d8a8e-5a50-4e48-897e-fe217a90488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.444980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d8a8e-5a50-4e48-897e-fe217a90488f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.445518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d8a8e-5a50-4e48-897e-fe217a90488f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.446360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d8a8e-5a50-4e48-897e-fe217a90488f None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.448016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d8a8e-5a50-4e48-897e-fe217a90488f None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.450341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d8a8e-5a50-4e48-897e-fe217a90488f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.453700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 61/119] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:53.455490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77ef72-ae06-407e-b4ae-fa80ff2254d0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.456247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77ef72-ae06-407e-b4ae-fa80ff2254d0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.456700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77ef72-ae06-407e-b4ae-fa80ff2254d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.474811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7f77ef72-ae06-407e-b4ae-fa80ff2254d0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:53.476007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7f77ef72-ae06-407e-b4ae-fa80ff2254d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:53.523925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018855-6fe2-4d81-85cc-1e16fb5f030e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.540846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 62/120] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:53.545659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa376e-83e2-43f1-bedd-079719030166 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.545953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa376e-83e2-43f1-bedd-079719030166 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.546766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa376e-83e2-43f1-bedd-079719030166 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.548236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 63/121] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:53.552883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa376e-83e2-43f1-bedd-079719030166 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.566683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.567560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.567846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.568120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.569407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.570875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.572693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 59/122] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:53.573313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.660816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa376e-83e2-43f1-bedd-079719030166 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.688595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa5ea-ecfe-4b3c-b50a-0b7eedab96d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.693767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccac0ea-9ad2-4c48-80fc-d587e72b0e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.694661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccac0ea-9ad2-4c48-80fc-d587e72b0e7f None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.694942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccac0ea-9ad2-4c48-80fc-d587e72b0e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.695213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccac0ea-9ad2-4c48-80fc-d587e72b0e7f None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/254e338acf294dff9fffc87141257a5d/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.697203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccac0ea-9ad2-4c48-80fc-d587e72b0e7f None admin] RBAC: Authorizing `identity:create_grant(project_id=8770b556c0c24d1b85b244d3070b0330, user_id=254e338acf294dff9fffc87141257a5d, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.698638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccac0ea-9ad2-4c48-80fc-d587e72b0e7f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.714006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 64/123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:30:53 2022] PUT /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/254e338acf294dff9fffc87141257a5d/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:30:53.715307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13904706-24c0-4693-9334-39565175f4ef None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.715654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13904706-24c0-4693-9334-39565175f4ef None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.715939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13904706-24c0-4693-9334-39565175f4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.737517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.738225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.738507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.738803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:53.739691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] RBAC: Adding query filter params (role.id=3867c58a744345018afcba3c9b9fb2e5, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=29d940305d7e4597aed3653717311f34) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:53.740297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:53.741804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:53.748199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 60/124] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/role_assignments?user.id=29d940305d7e4597aed3653717311f34&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=3867c58a744345018afcba3c9b9fb2e5 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:53.749139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.751434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-13904706-24c0-4693-9334-39565175f4ef None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:53.754235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-13904706-24c0-4693-9334-39565175f4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:53.895611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 65/125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:53.896533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13904706-24c0-4693-9334-39565175f4ef None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:53.920085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da326-0f84-4a26-8a25-eb32d98e8ae8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:53.991164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:53.991992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:53.992296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:53.992600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.005941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:54.014273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:54.015959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 61/126] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:54.016904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.075610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13904706-24c0-4693-9334-39565175f4ef None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.143728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aaa55-82a8-4ac7-a5a8-c13d310f8ad5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.144867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.145800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.146026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.146303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.147528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.149080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.150766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 66/127] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:53 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:54.151527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.192661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.193488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.193760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.194036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.212768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:54.228927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:54.229841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:54.230637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 62/128] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:54.231262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.328959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb351b-e7f9-40c1-a440-0ca11ad72458 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.367172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.368247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.368744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.369210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.370152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:54.370705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.372430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.382697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 67/129] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:54.383793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.466423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64bd1a-f38a-49d9-a938-d24aa7fb7fc4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.503181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a6387-34ae-4479-8ce5-f186340e0c32 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.539861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.541291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.541907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.542405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.543342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:54.543842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.545655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.554453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 68/130] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:54.555534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.556506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.557405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.557689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.557966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/dbe1e8845ac542ff9fb11a8bbaa70f79/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.564977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8770b556c0c24d1b85b244d3070b0330, user_id=dbe1e8845ac542ff9fb11a8bbaa70f79, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.566444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.593234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 63/131] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:30:54 2022] PUT /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/dbe1e8845ac542ff9fb11a8bbaa70f79/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:30:54.593836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.678335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a51981-c5a6-497d-b581-1de477a77bed None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.717083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.719128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.719616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.720076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.721975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.723649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.726165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 69/132] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:54.733508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.863112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae679-dfe7-4fe8-8d39-5e70d8170aa0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.872869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766e2c3-5560-4861-98dd-6ea673a7a49c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:54.909361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.910597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.911176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.911731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] PATH_INFO: `/v3/projects/46218928188b428a8d7b40eed5bf09aa` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.913288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=46218928188b428a8d7b40eed5bf09aa)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.913893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:54.914766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:54.915052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:54.915331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:54.915781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.917091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:54.918292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:54.919168 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 70/133] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/projects/46218928188b428a8d7b40eed5bf09aa => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:54.920043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 64/134] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:54.926344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:54.929829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:55.036894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20335d9-5544-4472-93f7-af67bb28588f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:55.060441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16301931-4534-49b0-84a6-8c22b13c9fb8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:55.080221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:55.080709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:55.081001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:55.081283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:55.084958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:55.086476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:55.088530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 65/135] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:54 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:55.094273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:55.107193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eb3f8-afe1-4928-8e83-bbaa7cbe85d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:55.108082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb3f8-afe1-4928-8e83-bbaa7cbe85d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:55.108391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb3f8-afe1-4928-8e83-bbaa7cbe85d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:55.109349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb3f8-afe1-4928-8e83-bbaa7cbe85d2 None admin] PATH_INFO: `/v3/projects/46218928188b428a8d7b40eed5bf09aa/users/dbe1e8845ac542ff9fb11a8bbaa70f79/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:55.117559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eb3f8-afe1-4928-8e83-bbaa7cbe85d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=46218928188b428a8d7b40eed5bf09aa, user_id=dbe1e8845ac542ff9fb11a8bbaa70f79, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:55.119381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eb3f8-afe1-4928-8e83-bbaa7cbe85d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:55.145613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 71/136] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:30:54 2022] PUT /identity/v3/projects/46218928188b428a8d7b40eed5bf09aa/users/dbe1e8845ac542ff9fb11a8bbaa70f79/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:30:55.251912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47db3d-1f6b-4cce-9948-ceb2f3887191 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:55.302589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60baf6-b250-4561-a532-465ee1968715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:55.303414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60baf6-b250-4561-a532-465ee1968715 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:55.303764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60baf6-b250-4561-a532-465ee1968715 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:55.304049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60baf6-b250-4561-a532-465ee1968715 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/dbe1e8845ac542ff9fb11a8bbaa70f79/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:55.306499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60baf6-b250-4561-a532-465ee1968715 None admin] RBAC: Authorizing `identity:create_grant(project_id=8770b556c0c24d1b85b244d3070b0330, user_id=dbe1e8845ac542ff9fb11a8bbaa70f79, role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:55.308006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60baf6-b250-4561-a532-465ee1968715 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:55.332022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 66/137] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:30:55 2022] PUT /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/dbe1e8845ac542ff9fb11a8bbaa70f79/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:30:55.837810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653e33-de51-4d4f-a933-e9f5423be6a4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:55.838202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653e33-de51-4d4f-a933-e9f5423be6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:55.838367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653e33-de51-4d4f-a933-e9f5423be6a4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:55.839744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 72/138] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:55.845274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:55.845828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:55.846306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:55.867680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:55.869196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:55.950465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 67/139] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:55.976129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653e33-de51-4d4f-a933-e9f5423be6a4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:56.081793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653e33-de51-4d4f-a933-e9f5423be6a4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:56.116926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45676c9e-94b4-433f-93ee-5ea2b3a900ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:56.118569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45676c9e-94b4-433f-93ee-5ea2b3a900ab None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:56.119049 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45676c9e-94b4-433f-93ee-5ea2b3a900ab None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:56.119527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45676c9e-94b4-433f-93ee-5ea2b3a900ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:56.121038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45676c9e-94b4-433f-93ee-5ea2b3a900ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:56.122710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45676c9e-94b4-433f-93ee-5ea2b3a900ab None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:56.125697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 73/140] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:56.131430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:56.239110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf60e8-65aa-4709-91a9-c3d3cc236cf6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:56.281310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb7450-774f-4160-8684-44fe2ddc3255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:56.282349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb7450-774f-4160-8684-44fe2ddc3255 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:56.282849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb7450-774f-4160-8684-44fe2ddc3255 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:56.283314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb7450-774f-4160-8684-44fe2ddc3255 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:56.284254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb7450-774f-4160-8684-44fe2ddc3255 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:56.285891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb7450-774f-4160-8684-44fe2ddc3255 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:56.330731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 68/141] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:30:56 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:30:56.855786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861f69-9600-484f-a1d6-a9537852ee2a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:56.856141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861f69-9600-484f-a1d6-a9537852ee2a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:56.856385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861f69-9600-484f-a1d6-a9537852ee2a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:56.857898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 74/142] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:56.863414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:56.863715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:56.863999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:56.883014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:56.884140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:56.887940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b638a2b-fd3b-46fe-9681-bc1bb52588fc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:56.888252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b638a2b-fd3b-46fe-9681-bc1bb52588fc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:56.888536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b638a2b-fd3b-46fe-9681-bc1bb52588fc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:56.890248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 75/143] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:30:56.896149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:56.896462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:56.896743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:56.917071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:56.918280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:56.966580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 69/144] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:30:57.001967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.008787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 76/145] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:57.150426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.205941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb54b50-40bf-40c2-acdb-8d5b1aedb1fb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.243176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.244385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.244924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.245399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.257523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:57.266283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:57.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:30:57.267907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:57.270383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 70/146] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:30:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:57.280323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.322796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa304ecb-29b1-4120-94ab-a791f756b799 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.398392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06840395-3561-4a42-a34d-5e1d01f0a1d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.421485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.422479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.422748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.423023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.438869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.439910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.440422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.440900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.442041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:57.442551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:57.444250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:57.454206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:57.455455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 71/147] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:57.462226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:57.462628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:57.463471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:30:57.464289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 77/148] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:57.470464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.592611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c31b8c-2133-44e3-bc69-a21647ced031 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.640978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.641908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.642197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.642476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] PATH_INFO: `/v3/users/254e338acf294dff9fffc87141257a5d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.643832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] RBAC: Authorizing `identity:get_user(user_id=254e338acf294dff9fffc87141257a5d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:57.645353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:57.647200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 72/149] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/users/254e338acf294dff9fffc87141257a5d => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:57.652597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.693438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993e1a0-04af-4074-ab7c-6abedd304b07 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.737197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.737875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.738158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.738434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.739198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:57.739518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:57.741196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:57.751774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 78/150] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:57.757407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.798677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5d6d3-5b6a-4ff3-b0d8-9663cfeedb17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.844963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.845684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.845973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.846251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.866003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20add7d-8922-4b5f-be97-ba549c0c61b9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:57.899800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ac80d-f4f0-48a2-afca-0637f67e406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:57.900715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ac80d-f4f0-48a2-afca-0637f67e406e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.900999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ac80d-f4f0-48a2-afca-0637f67e406e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.901268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ac80d-f4f0-48a2-afca-0637f67e406e None admin] PATH_INFO: `/v3/users/29d940305d7e4597aed3653717311f34` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.902526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ac80d-f4f0-48a2-afca-0637f67e406e None admin] RBAC: Authorizing `identity:get_user(user_id=29d940305d7e4597aed3653717311f34)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:57.903972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ac80d-f4f0-48a2-afca-0637f67e406e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:57.906345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 79/151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/users/29d940305d7e4597aed3653717311f34 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:57.907908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f0d10-8fe6-4436-ad8d-5a40f70a9308 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:57.912235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f0d10-8fe6-4436-ad8d-5a40f70a9308 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:57.912573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f0d10-8fe6-4436-ad8d-5a40f70a9308 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:57.913948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 80/152] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:57.917490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f0d10-8fe6-4436-ad8d-5a40f70a9308 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:57.938582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:57.940486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:58.024945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f0d10-8fe6-4436-ad8d-5a40f70a9308 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:58.062839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0281dd-ca7f-4663-ba82-665e718b8c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:58.063700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0281dd-ca7f-4663-ba82-665e718b8c72 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.063977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0281dd-ca7f-4663-ba82-665e718b8c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.064279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0281dd-ca7f-4663-ba82-665e718b8c72 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.069839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0281dd-ca7f-4663-ba82-665e718b8c72 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:58.071276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0281dd-ca7f-4663-ba82-665e718b8c72 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:58.073021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 81/153] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:58.074187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.074468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.074743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.100784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:58.101974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:58.131295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 73/154] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 17:30:58.132988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:58.173800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 82/155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:58.174652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:58.284913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77392938-81f6-4ddc-b439-de8d1c1218f7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:58.322109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:58.322989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.323272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.323545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/29d940305d7e4597aed3653717311f34/roles/3867c58a744345018afcba3c9b9fb2e5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.325561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] RBAC: Authorizing `identity:create_grant(project_id=8770b556c0c24d1b85b244d3070b0330, user_id=29d940305d7e4597aed3653717311f34, role_id=3867c58a744345018afcba3c9b9fb2e5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:58.327928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:58.353610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 74/156] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:30:58 2022] PUT /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330/users/29d940305d7e4597aed3653717311f34/roles/3867c58a744345018afcba3c9b9fb2e5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:30:58.354751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d660-4348-4ec8-ab43-1aa36514d93e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:58.356463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:58.436083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:58.437822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.438103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.438375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.445177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:58.446655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:58.448547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 83/157] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:58 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:58.455519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:58.495921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c432-aa9f-44d9-8c67-0ed0acb52322 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:58.586983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:58.587724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.588002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.588302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.615706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:30:58.634068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:30:58.634978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:30:58.635737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 75/158] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:30:58.645915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:58.711691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4aae0-3949-4f90-a8ef-c20dd60a24e9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:58.762877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297f924-79f1-4f3f-85ce-86f07cf64dbf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:58.788373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:58.789028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.789304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.789575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.790411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=254e338acf294dff9fffc87141257a5d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:58.790709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:58.792140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:58.797276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:58.798123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:58.798446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:58.798751 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:58.799825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:58.800383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:58.801806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:58.810413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 84/159] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:58 2022] GET /identity/v3/role_assignments?user.id=254e338acf294dff9fffc87141257a5d&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:58.810786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 76/160] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:30:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:58.817347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:58.978769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b8560-6593-46e9-90a0-ecfb5f91344b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:59.021986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:59.023324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:59.023779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:59.024248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:59.027023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:59.028697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:59.031282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 85/161] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:30:58 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:59.037186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:59.157294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec056fa7-379f-4c36-a58f-3cbbcf7aaa29 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:59.194069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:59.195047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:59.195511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:59.196007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:59.272586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:59.274353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:59.376356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 77/162] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:30:59.382274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:59.495082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073656e-71ac-470a-9786-5faa60c3e992 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:59.538213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643210ce-28b5-4f21-8df8-54debec427a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:59.539349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643210ce-28b5-4f21-8df8-54debec427a3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:59.539803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643210ce-28b5-4f21-8df8-54debec427a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:59.540273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643210ce-28b5-4f21-8df8-54debec427a3 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:59.543526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643210ce-28b5-4f21-8df8-54debec427a3 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:59.545187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643210ce-28b5-4f21-8df8-54debec427a3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:59.547372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 86/163] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:30:59 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:30:59.553261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:59.669473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f814f6-00e4-415e-a512-70cfec0934ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:30:59.704763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:30:59.705428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:59.705709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:59.705984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:59.706823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=dbe1e8845ac542ff9fb11a8bbaa70f79) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:30:59.707132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:30:59.708677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940d2e6-5d1c-4519-b3d2-798ffe90924c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:30:59.715306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 78/164] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:30:59 2022] GET /identity/v3/role_assignments?user.id=dbe1e8845ac542ff9fb11a8bbaa70f79&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:30:59.721861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c18511-ec9b-4190-9b1b-fe952956e2cf None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:59.722239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c18511-ec9b-4190-9b1b-fe952956e2cf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:59.722570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c18511-ec9b-4190-9b1b-fe952956e2cf None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:59.724191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 87/165] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:30:59.730045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:30:59.730332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:30:59.730600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:30:59.753607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:30:59.754883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:30:59.832382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 79/166] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:30:59.856394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c18511-ec9b-4190-9b1b-fe952956e2cf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:30:59.971249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c18511-ec9b-4190-9b1b-fe952956e2cf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.009419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58bcd8-a65b-4b9e-afa9-5d5aa2ce4aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.010229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58bcd8-a65b-4b9e-afa9-5d5aa2ce4aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.010512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58bcd8-a65b-4b9e-afa9-5d5aa2ce4aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.010845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58bcd8-a65b-4b9e-afa9-5d5aa2ce4aa3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.032530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58bcd8-a65b-4b9e-afa9-5d5aa2ce4aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7d58bcd8-a65b-4b9e-afa9-5d5aa2ce4aa3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:00.040709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:00.041639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:00.042411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 88/167] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:30:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:00.048272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.056198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6629f2-ebd3-4a1c-9f57-eef010d51b99 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.056198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6629f2-ebd3-4a1c-9f57-eef010d51b99 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.056198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6629f2-ebd3-4a1c-9f57-eef010d51b99 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.057274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 89/168] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:00.063352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.063857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.064352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.089269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:00.090793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:00.154271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449b59-b3af-40c8-a414-ba13a8afbb35 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.176324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 90/169] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:00.195543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.196281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.196564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.196837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.197581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:00.197886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:00.199267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:00.208128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 80/170] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:00.213790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.219313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.331512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b21d8-0f52-4a46-ac81-fdfd3ed0fa1a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.346500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0628b2b-fb0a-4d13-ae85-1d9688609ac0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.389040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.389902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.390294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.390727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.391779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.393407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.393887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.394356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.395786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:00.397734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:00.400046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 91/171] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:00.403952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:00.409140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:00.412482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:00.413377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:00.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 81/172] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:00.419318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.527149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0846664-c890-49e8-96fa-85ecc2ab12a4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.569764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.571440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.572212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.572665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.580931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825a38b-69b6-48a3-a410-a2ce92c713a4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.629753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.630439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.630723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.631172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.632025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:00.632365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:00.634246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:00.643903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 82/173] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:00.649632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.658773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:00.660275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:00.765624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ffc14-6c8d-4bbd-ac65-01c306e926fa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:00.773461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 92/174] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:00.781803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.827151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:00.828929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:00.829407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:00.829872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] PATH_INFO: `/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:00.831362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] RBAC: Authorizing `identity:get_user(user_id=dbe1e8845ac542ff9fb11a8bbaa70f79)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:00.834065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:00.836574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 83/175] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/users/dbe1e8845ac542ff9fb11a8bbaa70f79 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:00.842131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:00.955907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39048683-4237-48d3-8b7d-d161bc672b57 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:01.008591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8eb570-3aaf-459c-892f-1d50fd7ff7be None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:01.025467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:01.026632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.027111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.027581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] PATH_INFO: `/v3/projects/46218928188b428a8d7b40eed5bf09aa` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.030853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] RBAC: Authorizing `identity:get_project(project_id=46218928188b428a8d7b40eed5bf09aa)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:01.032588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:01.034539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 93/176] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/projects/46218928188b428a8d7b40eed5bf09aa => generated 312 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:01.040933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:01.091784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:01.092890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.093734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.094209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.232678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:01.235605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:01.247458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37baf962-023a-4af1-b6bf-b24b64884a9b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:01.306418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:01.307461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.307938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.308432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.309470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=46218928188b428a8d7b40eed5bf09aa, user.id=dbe1e8845ac542ff9fb11a8bbaa70f79) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:01.309996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:01.312152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:01.321134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 94/177] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:01 2022] GET /identity/v3/role_assignments?user.id=dbe1e8845ac542ff9fb11a8bbaa70f79&scope.project.id=46218928188b428a8d7b40eed5bf09aa&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:01.381419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 84/178] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:01.389300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:01.514001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12bb4e-0d78-4f5b-b1a3-70ded83d04da None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:01.562049 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacd4b1-6ac5-4c3c-b375-a520cbc8da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:01.563263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd4b1-6ac5-4c3c-b375-a520cbc8da62 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.563741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd4b1-6ac5-4c3c-b375-a520cbc8da62 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.564254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd4b1-6ac5-4c3c-b375-a520cbc8da62 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.568915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacd4b1-6ac5-4c3c-b375-a520cbc8da62 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:01.570606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacd4b1-6ac5-4c3c-b375-a520cbc8da62 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:01.572731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 95/179] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:01 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:01.579000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:01.679509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a3afd-bb5b-4acb-a08e-3befce99a699 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.679824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a3afd-bb5b-4acb-a08e-3befce99a699 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.680059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a3afd-bb5b-4acb-a08e-3befce99a699 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.681507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 96/180] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:01.693904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e2382-487a-4656-8756-332db9d2564b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.694245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e2382-487a-4656-8756-332db9d2564b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.694462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e2382-487a-4656-8756-332db9d2564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.719864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eb9e2382-487a-4656-8756-332db9d2564b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:01.721195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eb9e2382-487a-4656-8756-332db9d2564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:01.729750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27157f2-e29a-4fd0-8d88-15440c9f4571 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:01.808376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:01.809007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.809327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.809645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.810491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] RBAC: Adding query filter params (role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=dbe1e8845ac542ff9fb11a8bbaa70f79) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:01.810799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:01.812541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:01.824135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 85/181] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:01 2022] GET /identity/v3/role_assignments?user.id=dbe1e8845ac542ff9fb11a8bbaa70f79&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:01.855249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 97/182] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:01.899067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:01.943982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788046c-3d28-4cb9-b990-366ffd1848ec None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.944297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788046c-3d28-4cb9-b990-366ffd1848ec None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.944577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788046c-3d28-4cb9-b990-366ffd1848ec None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.945941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 98/183] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:01.957296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31d49-2fc9-4d22-a68c-c0b39aef816d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:01.957586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31d49-2fc9-4d22-a68c-c0b39aef816d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:01.957867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31d49-2fc9-4d22-a68c-c0b39aef816d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:01.979978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-55f31d49-2fc9-4d22-a68c-c0b39aef816d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:01.981686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-55f31d49-2fc9-4d22-a68c-c0b39aef816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:02.026983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3e00f-a19c-41dc-adf1-38d9c9201cb4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.076927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fef291-b8fc-4447-93bd-300c836988ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.077724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef291-b8fc-4447-93bd-300c836988ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.078001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef291-b8fc-4447-93bd-300c836988ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.078268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef291-b8fc-4447-93bd-300c836988ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.101069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fef291-b8fc-4447-93bd-300c836988ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f0fef291-b8fc-4447-93bd-300c836988ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:02.108963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:02.109915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:02.110702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 86/184] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:02.112109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19c152-2ce6-4119-a2d6-39b154843c2c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.112109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19c152-2ce6-4119-a2d6-39b154843c2c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.112301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19c152-2ce6-4119-a2d6-39b154843c2c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.113575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 87/185] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:02.115707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 99/186] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:02.120153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19c152-2ce6-4119-a2d6-39b154843c2c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.121250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.121781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.122264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.147303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:02.148979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:02.244733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 100/187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:02.245935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.278227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19c152-2ce6-4119-a2d6-39b154843c2c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.311009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.312345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.312716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.313126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.313937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:02.314309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.315750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.324414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 88/188] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:02.325111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.351146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b7b4-7db5-41f4-a0c0-aa772a708927 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.383361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.384180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.384467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.384740 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.395178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:02.403131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:02.404001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:02.404821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 101/189] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:02.405509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.418089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7e7a1-a528-42cb-991f-124a007fce4e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.450816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bea21-e344-4ec4-a1ca-b252d4adb651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.451599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bea21-e344-4ec4-a1ca-b252d4adb651 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.451875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bea21-e344-4ec4-a1ca-b252d4adb651 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.452191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bea21-e344-4ec4-a1ca-b252d4adb651 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.453431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bea21-e344-4ec4-a1ca-b252d4adb651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.454864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bea21-e344-4ec4-a1ca-b252d4adb651 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.456521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 89/190] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:02.457679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e4f8-247b-4ed2-bb30-c819b5a973af None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.457959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e4f8-247b-4ed2-bb30-c819b5a973af None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.458236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e4f8-247b-4ed2-bb30-c819b5a973af None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.459590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 90/191] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:02.460121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812e4f8-247b-4ed2-bb30-c819b5a973af None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.499915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe81396-d599-443f-ae65-9051161eae90 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.530905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.531772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.532053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.532361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] PATH_INFO: `/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.544815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] RBAC: Authorizing `identity:get_user(user_id=f0d0beaec57e4af5ba5e5146dd9d2acb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.546849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.552954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 102/192] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:02.552954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.553848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812e4f8-247b-4ed2-bb30-c819b5a973af None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.587095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.587754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.588032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.588331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.589072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:02.589376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.590733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7622e-c1f9-41e1-be71-4f4d5ac97486 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.598925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 91/193] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:02.600109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.600419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.600693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.616739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:02.617885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:02.651058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d75da-02e0-4f8f-aec0-4c7c8fa725ad None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.679563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 92/194] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:02.680538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.695025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.695722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.696021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.696320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.697036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.698484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.720410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 103/195] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:02 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:02.725212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.788429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286227dc-c512-4922-abce-a9fc2c456dfe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.825605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.826256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.826532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.826802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.833030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faca30e-ad63-4dc8-a638-ee7f5482600b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:02.870916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:02.871726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:02.872005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:02.872313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] PATH_INFO: `/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:02.873665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] RBAC: Authorizing `identity:get_user(user_id=f0d0beaec57e4af5ba5e5146dd9d2acb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.875584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.877326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 104/196] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:02.878101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:02.897228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:02.898967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:02.989085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4fcb4-7755-4a31-8f22-9d69241c2b1b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.015638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 93/197] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:03.017499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.025913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.026715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.026997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.027274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.040065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:03.048140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:03.049159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:03.049929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 105/198] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:03.051276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.134062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4630a0-612b-4879-ab8a-c97a73cfe76f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.163263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0c2b9-2a57-4689-a025-1fd4f64ff959 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.169603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.170253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.170536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.170926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.198804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.200037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.200551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.201011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.203074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.204760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.206720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 106/199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:03.207791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.248943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.250577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.335256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72463d7f-45e7-44bf-8dd0-5dac35d80e40 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.349000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 94/200] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:03.349844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.382082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.382800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.383085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.383359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.384187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:03.384441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.385808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.394916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 107/201] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:03.395556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.459302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4690-709d-40c6-aafa-0fd3dea1cf50 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.499863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b75f0-9803-412f-805c-7bba61c22fcd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.502803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.503665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.504149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.504621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.505624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=f0d0beaec57e4af5ba5e5146dd9d2acb) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:03.506129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.508194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.520572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 95/202] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/role_assignments?user.id=f0d0beaec57e4af5ba5e5146dd9d2acb&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:03.521787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.534625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.535628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.535978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.536341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.537626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.539227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.544191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 108/203] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:03.547353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.672692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7f44-9a21-4bec-8ce6-1104b1d66fc3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.673572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2d4b6-e06d-4313-90d2-0e07b23f2dd5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.710313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.711192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.711467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.711739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] PATH_INFO: `/v3/users/29d940305d7e4597aed3653717311f34` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.712993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] RBAC: Authorizing `identity:get_user(user_id=29d940305d7e4597aed3653717311f34)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.714529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.716232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 96/204] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/users/29d940305d7e4597aed3653717311f34 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:03.722559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:03.751113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.751792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.752076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.752378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.753294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] RBAC: Adding query filter params (role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=f0d0beaec57e4af5ba5e5146dd9d2acb) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:03.753663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.755130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:03.762661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 109/205] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/role_assignments?user.id=f0d0beaec57e4af5ba5e5146dd9d2acb&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:03.836608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c30cc-0170-47e7-9ff2-0f473fb20692 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:03.877347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:03.878020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:03.878303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:03.878578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:03.948557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:03.949956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:04.051010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 97/206] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 17:31:04.056626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:04.169369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab738-84eb-4622-be1e-1b89fc7dc986 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:04.218703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75c035-f375-4ab8-812e-535d56be0377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:04.219536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75c035-f375-4ab8-812e-535d56be0377 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:04.219817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75c035-f375-4ab8-812e-535d56be0377 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:04.220092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75c035-f375-4ab8-812e-535d56be0377 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:04.221343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75c035-f375-4ab8-812e-535d56be0377 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:04.222781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75c035-f375-4ab8-812e-535d56be0377 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:04.224497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 110/207] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:04 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:04.230317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:04.335776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a1754-9b38-4a9e-b767-2b968d01d417 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:04.370698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:04.371482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:04.371642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:04.371916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:04.372781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] RBAC: Adding query filter params (role.id=3867c58a744345018afcba3c9b9fb2e5, scope.project.id=8770b556c0c24d1b85b244d3070b0330, user.id=29d940305d7e4597aed3653717311f34) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:04.373087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:04.374569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f90e5-31a1-4b32-9ad0-9d0c6e50a6a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:04.380583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 98/208] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:04 2022] GET /identity/v3/role_assignments?user.id=29d940305d7e4597aed3653717311f34&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=3867c58a744345018afcba3c9b9fb2e5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:05.380373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54e79-b191-4814-ab3b-77fd88b65ea1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.381052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54e79-b191-4814-ab3b-77fd88b65ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.381052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54e79-b191-4814-ab3b-77fd88b65ea1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.382274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 111/209] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:05.387334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.387627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.387904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.388985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa19f4-3fcb-4d59-93db-d9aa16c81a53 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.390440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa19f4-3fcb-4d59-93db-d9aa16c81a53 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.390815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa19f4-3fcb-4d59-93db-d9aa16c81a53 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.392431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 99/210] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:05.397897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.398420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.398932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.406353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:05.407548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:05.417413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:05.418848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:05.476970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 112/211] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:05.505665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:05.511547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 100/212] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:05.544409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:05.615335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256fafe-35d0-4980-8b74-04600862f007 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:05.651373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:05.652410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.652833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.653246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.664215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:05.667814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe54904-ccaa-45a8-af96-84f6722d32d6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:05.673340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:05.674301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:05.675084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:05.677412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 113/213] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:05.683069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:05.713187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:05.714354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.714638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.714938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.716377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:05.718070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:05.720693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 101/214] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:05.725798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:05.787243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e2172-df8f-4d35-a41a-8d574fa29e22 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:05.824378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:05.825059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.825344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.825623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.826388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:05.826701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:05.828230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:05.830551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56940b-3adc-47e7-8fbc-3d0261c1158d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:05.836936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 114/215] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:05.843136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:05.872927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:05.874352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:05.874835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:05.875528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:05.876568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:05.878811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:05.905590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 102/216] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:05 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:31:05.962857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32283b0a-2d0f-493d-b484-3f8797b01807 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:06.002227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1477642-fa87-4e0e-95df-10aefbde6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:06.003583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477642-fa87-4e0e-95df-10aefbde6c7a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.004087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477642-fa87-4e0e-95df-10aefbde6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.004598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1477642-fa87-4e0e-95df-10aefbde6c7a None admin] PATH_INFO: `/v3/users/a68875542cac48df81d344fdc20bd3a6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.021809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1477642-fa87-4e0e-95df-10aefbde6c7a None admin] RBAC: Authorizing `identity:get_user(user_id=a68875542cac48df81d344fdc20bd3a6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:06.023650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1477642-fa87-4e0e-95df-10aefbde6c7a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:06.025785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 115/217] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:05 2022] GET /identity/v3/users/a68875542cac48df81d344fdc20bd3a6 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:06.032200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:06.142380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cf12d-68c4-4689-8838-fb88cd739239 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:06.175869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:06.176476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.176762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.177035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.243099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e98c-b7ad-443a-bbd2-1ab671fbdcb9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.243414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e98c-b7ad-443a-bbd2-1ab671fbdcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.243657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e98c-b7ad-443a-bbd2-1ab671fbdcb9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.245031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 116/218] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:06.253542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:06.253693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:06.255828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.256182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.260758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.285414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:06.287210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:06.383582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 103/219] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:06.391960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:06.394032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 117/220] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:06.425496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:06.536769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776d941-82cc-4856-adbe-0ed1c3f7fcd1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:06.563490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811908ed-a13d-4ec1-b24b-4194759cd639 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:06.576142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705101a8-3a0e-4bfb-b57b-7ac4c3cf4260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:06.577285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705101a8-3a0e-4bfb-b57b-7ac4c3cf4260 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.577619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705101a8-3a0e-4bfb-b57b-7ac4c3cf4260 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.578114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705101a8-3a0e-4bfb-b57b-7ac4c3cf4260 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.579462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705101a8-3a0e-4bfb-b57b-7ac4c3cf4260 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:06.581566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705101a8-3a0e-4bfb-b57b-7ac4c3cf4260 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:06.586390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 104/221] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:06 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:06.587705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a44dc-b077-473d-a9ca-c5b487bfb2a2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.588121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a44dc-b077-473d-a9ca-c5b487bfb2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.588490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a44dc-b077-473d-a9ca-c5b487bfb2a2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.590060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 105/222] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:06.595808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a44dc-b077-473d-a9ca-c5b487bfb2a2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:06.614426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3d7e2-ec9d-460e-911a-9bafae1182d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:06.615479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3d7e2-ec9d-460e-911a-9bafae1182d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.615761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3d7e2-ec9d-460e-911a-9bafae1182d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.616033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3d7e2-ec9d-460e-911a-9bafae1182d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.632192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3d7e2-ec9d-460e-911a-9bafae1182d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e6c3d7e2-ec9d-460e-911a-9bafae1182d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:06.639561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:06.641275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 118/223] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:06.642067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.642786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.642786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.664197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:06.664197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:06.771862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 119/224] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:06.784187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:06.833989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a44dc-b077-473d-a9ca-c5b487bfb2a2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:06.875701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:06.876394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.876686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.877375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.878354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=a68875542cac48df81d344fdc20bd3a6) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:06.878798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:06.880514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:06.887109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 106/225] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:06 2022] GET /identity/v3/role_assignments?user.id=a68875542cac48df81d344fdc20bd3a6&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:06.892365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:06.953857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca911f-0356-4d67-9c9c-9d3bf9a28669 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:06.995199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:06.995883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:06.996179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:06.996460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:06.997224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:06.997528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:06.999515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.018094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 120/226] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:07.024325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:07.122436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5381e-782f-4c52-a792-0e4600f50f30 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.188610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.189424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.189702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.189972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.210042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:07.218811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:07.219685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:07.220563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:07.232282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 107/227] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:07.235732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85579c-1f5d-45e1-8744-91f5ccd7688b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.237778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:07.305807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.307054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.307561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.308032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] PATH_INFO: `/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.309451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] RBAC: Authorizing `identity:get_user(user_id=f0d0beaec57e4af5ba5e5146dd9d2acb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:07.311096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.313526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 121/228] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:07 2022] GET /identity/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:07.320926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:07.419247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef10c-de64-4604-bbbe-68adac6dd97a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.491129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.491810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.492093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.492394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.494069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:07.494379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:07.495781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.515667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 108/229] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:07.525170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:07.528366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5cfdf-317e-463d-b555-7d092a2d22cc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.601825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.602957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.603384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.603823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.605180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:07.606867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.616188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 122/230] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:07 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:07.617564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:07.628144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd233b5-6dfc-4abb-9b86-86f0fad5d293 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.666239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.667118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.667398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.667669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] PATH_INFO: `/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.669062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] RBAC: Authorizing `identity:get_user(user_id=f0d0beaec57e4af5ba5e5146dd9d2acb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:07.670567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.672966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 109/231] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:07 2022] GET /identity/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:07.681069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:07.810588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86479f49-9bb2-49d6-a27d-8220ef0632cd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.844747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.845972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.846455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.846927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/f0d0beaec57e4af5ba5e5146dd9d2acb/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.847691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd71129-e18b-46ea-8fdb-bc9c550735a8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:07.849352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7b5bc85b1406083bf0c40dee773d1, user_id=f0d0beaec57e4af5ba5e5146dd9d2acb, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:07.850981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.865907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 123/232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:07 2022] PUT /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/f0d0beaec57e4af5ba5e5146dd9d2acb/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:07.899555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51c0e8-2076-4409-aaac-31efa45058b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:07.900898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51c0e8-2076-4409-aaac-31efa45058b4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:07.901382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51c0e8-2076-4409-aaac-31efa45058b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:07.901863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51c0e8-2076-4409-aaac-31efa45058b4 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:07.903412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51c0e8-2076-4409-aaac-31efa45058b4 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:07.905794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51c0e8-2076-4409-aaac-31efa45058b4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:07.911248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 110/233] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:07 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:07.914553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:08.026314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0389c-bbd1-488f-aaf6-7f80c9a45f1b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:08.061064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26c050-6691-42c0-9797-5683451b094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:08.062870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c050-6691-42c0-9797-5683451b094a None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:08.063349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c050-6691-42c0-9797-5683451b094a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:08.063818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c050-6691-42c0-9797-5683451b094a None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/f0d0beaec57e4af5ba5e5146dd9d2acb/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:08.066261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26c050-6691-42c0-9797-5683451b094a None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7b5bc85b1406083bf0c40dee773d1, user_id=f0d0beaec57e4af5ba5e5146dd9d2acb, role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:08.067904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26c050-6691-42c0-9797-5683451b094a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:08.095574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 124/234] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:07 2022] PUT /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/f0d0beaec57e4af5ba5e5146dd9d2acb/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:08.731541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda5e68-a8a6-404e-bebb-da43eb964007 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:08.732602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda5e68-a8a6-404e-bebb-da43eb964007 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:08.733092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda5e68-a8a6-404e-bebb-da43eb964007 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:08.736422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 111/235] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:08.742213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d288a9-c4e5-4ced-a224-37be2da15dfe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:08.743178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d288a9-c4e5-4ced-a224-37be2da15dfe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:08.743657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d288a9-c4e5-4ced-a224-37be2da15dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:08.762548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a4d288a9-c4e5-4ced-a224-37be2da15dfe None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:08.764154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a4d288a9-c4e5-4ced-a224-37be2da15dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:08.859224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 125/236] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:08.898227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda5e68-a8a6-404e-bebb-da43eb964007 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:08.948998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c0993-6af5-42c1-bc3e-aad6dea37782 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:08.949388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c0993-6af5-42c1-bc3e-aad6dea37782 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:08.949774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c0993-6af5-42c1-bc3e-aad6dea37782 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:08.951143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 126/237] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:08.956895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:08.957183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:08.957465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:08.975727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:08.977233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:09.055490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda5e68-a8a6-404e-bebb-da43eb964007 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.103117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:09.105265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 127/238] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:09.106399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.106899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.107385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.112940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:09.114408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:09.116358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 112/239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:09.121636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.129781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.229129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c6ee5-5205-48a7-88b8-6329e0721a15 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.263727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:09.264452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.265216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.265216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.265691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:09.267123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:09.292470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 128/240] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:09 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:09.328274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a012f5-9010-4390-adbf-6df4c577b07e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.367208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b21b27-f943-493b-949d-04e4e533eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:09.368455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21b27-f943-493b-949d-04e4e533eefd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.368946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21b27-f943-493b-949d-04e4e533eefd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.369448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21b27-f943-493b-949d-04e4e533eefd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.381695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b21b27-f943-493b-949d-04e4e533eefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-10b21b27-f943-493b-949d-04e4e533eefd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:09.390430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:09.391305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:09.392104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:09.395056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 113/241] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:09.401328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.451609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c5671-5caa-44aa-9c90-4ca9dee7c35f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.452009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c5671-5caa-44aa-9c90-4ca9dee7c35f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.452417 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c5671-5caa-44aa-9c90-4ca9dee7c35f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.454499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 114/242] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:09.472548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.472876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.473113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.502260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:09.503386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:09.595385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65208d2-efe6-4a73-8e09-b5d41a8f1f98 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.635278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 115/243] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:09.649827 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:09.650825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.651310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.651861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.652897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:09.653476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:09.655607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:09.660285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.666125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 129/244] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:09.673803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.790992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fbc4f-40a8-4d5e-a5df-8c91f423e6fa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.803427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f12dd-0f44-4b18-8c8c-1385fb0c73a3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.830780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:09.831578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.831859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.832132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.843953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:09.848362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:09.849487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:09.849983 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:09.850473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] PATH_INFO: `/v3/users/56238a3816404b33a4dd4f1c0cea0ec8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:09.854522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:09.855414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:09.856225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 116/245] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:09.865460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.870028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] RBAC: Authorizing `identity:get_user(user_id=56238a3816404b33a4dd4f1c0cea0ec8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:09.871709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:09.874150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 130/246] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:09 2022] GET /identity/v3/users/56238a3816404b33a4dd4f1c0cea0ec8 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:09.880111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:09.990274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2872b97-5539-4c32-96c6-6ae8e8ef26f2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:09.997822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645e863-bab4-4595-af8c-ae8f642edb44 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:10.029187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.029989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.030846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.031198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.045887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.046835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.047466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.047924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.049265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:10.049752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.051284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.060740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 117/247] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:10.065910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:10.115604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.120184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.216586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 131/248] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 17:31:10.222383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:10.234910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1bb78-5f83-473b-9bfd-eb4f57894972 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:10.316944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.317747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.318029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.318302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] PATH_INFO: `/v3/users/a68875542cac48df81d344fdc20bd3a6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.319519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] RBAC: Authorizing `identity:get_user(user_id=a68875542cac48df81d344fdc20bd3a6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.320970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.322908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 118/249] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:10 2022] GET /identity/v3/users/a68875542cac48df81d344fdc20bd3a6 => generated 297 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:10.323642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259928-0c98-44d7-8dff-cb98b127553e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:10.328837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:10.358774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.359655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.359938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.360236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.361787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.363457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.365617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 132/250] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:10 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:10.371874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:10.485824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2441b30-53e3-46be-806c-69f363bcd3a9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:10.525114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.525830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.526111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.526381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.531849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] RBAC: Adding query filter params (role.id=3867c58a744345018afcba3c9b9fb2e5, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=56238a3816404b33a4dd4f1c0cea0ec8) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:10.531849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.531849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.537848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 133/251] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:10 2022] GET /identity/v3/role_assignments?user.id=56238a3816404b33a4dd4f1c0cea0ec8&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=3867c58a744345018afcba3c9b9fb2e5 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:10.594411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1b93-8c98-4ad5-90f1-50458a76662e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:10.653866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a780489-b615-461b-a0a3-0984d630ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.655171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780489-b615-461b-a0a3-0984d630ca62 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.655656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780489-b615-461b-a0a3-0984d630ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.656128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780489-b615-461b-a0a3-0984d630ca62 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.657697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a780489-b615-461b-a0a3-0984d630ca62 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.659402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a780489-b615-461b-a0a3-0984d630ca62 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.661732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 119/252] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:10 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:10.667177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:10.803564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1083c-6785-4066-8277-233d20af58a1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:10.840523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cb9a0-5eb1-46a7-83da-d94ae3a49275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:10.841441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb9a0-5eb1-46a7-83da-d94ae3a49275 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:10.841722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb9a0-5eb1-46a7-83da-d94ae3a49275 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:10.841999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb9a0-5eb1-46a7-83da-d94ae3a49275 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/a68875542cac48df81d344fdc20bd3a6/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:10.845078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cb9a0-5eb1-46a7-83da-d94ae3a49275 None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7b5bc85b1406083bf0c40dee773d1, user_id=a68875542cac48df81d344fdc20bd3a6, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:10.846520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cb9a0-5eb1-46a7-83da-d94ae3a49275 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:10.871590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 134/253] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:10 2022] PUT /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/a68875542cac48df81d344fdc20bd3a6/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:11.226045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fb0d8-0f7b-46f9-a4e1-8c8819da4010 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:11.227053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fb0d8-0f7b-46f9-a4e1-8c8819da4010 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:11.227534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fb0d8-0f7b-46f9-a4e1-8c8819da4010 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:11.229426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 120/254] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:11.241080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:11.241369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:11.241644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:11.265958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:11.267160 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:11.342595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 135/255] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:11.378992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fb0d8-0f7b-46f9-a4e1-8c8819da4010 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:11.524276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fb0d8-0f7b-46f9-a4e1-8c8819da4010 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:11.569906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d63728-91c5-48b9-ad8f-5f9daac8d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:11.570706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d63728-91c5-48b9-ad8f-5f9daac8d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:11.570988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d63728-91c5-48b9-ad8f-5f9daac8d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:11.571264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d63728-91c5-48b9-ad8f-5f9daac8d1dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:11.595415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d63728-91c5-48b9-ad8f-5f9daac8d1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-68d63728-91c5-48b9-ad8f-5f9daac8d1dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:11.603110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:11.603993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:11.604813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 121/256] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:11.613295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:11.723741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d820a-5830-4bb5-bb06-42f23b91558a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:11.724062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d820a-5830-4bb5-bb06-42f23b91558a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:11.724310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d820a-5830-4bb5-bb06-42f23b91558a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:11.725908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 122/257] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:11.735213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:11.735503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:11.735778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:11.748334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8268-9b55-4cfc-98f2-dcbf84b206e5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:11.782003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:11.783175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:11.784949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:11.785794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:11.786125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:11.786403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:11.787158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:11.787460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:11.789721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:11.798716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 136/258] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:11.806164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:11.954636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 123/259] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:11.991864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:11.993317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128a3a2-e91e-47eb-bab6-3ec2e236b331 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.069555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.070360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.070642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.070916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] PATH_INFO: `/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.072136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] RBAC: Authorizing `identity:get_user(user_id=f0d0beaec57e4af5ba5e5146dd9d2acb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.073602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.084511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 137/260] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:11 2022] GET /identity/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:12.085428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.103206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75764ce0-f0f3-45e8-aed6-9977339ca191 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.141888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.142697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.143024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.143307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.154485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:12.164591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:12.165506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:12.166313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 124/261] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:12.173207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.279478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7d574-cdcf-42e6-b1c3-920ccbbafccf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.321847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.322431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.322717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.322988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.323735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:12.324040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.325460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.339883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 125/262] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:12.342134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec62e-0fe6-4bc4-b709-85c0574d7d2a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.345626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.429445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.430104 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.430386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.430663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.455614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d6ed6-e180-42eb-9798-a15d3385ee14 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.489511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.490688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.491016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.491293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] PATH_INFO: `/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.492522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] RBAC: Authorizing `identity:get_user(user_id=f0d0beaec57e4af5ba5e5146dd9d2acb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.494913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.497373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 126/263] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/users/f0d0beaec57e4af5ba5e5146dd9d2acb => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:12.503306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.603407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddbdda-caf3-4a46-882c-5aa0948827cd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.604960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.606427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.637042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.637692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.637979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.638259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.711219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.712730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.814275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 138/265] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:12.814787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 127/265] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:12.819298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.819459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.928187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f5d9-2de3-4fa3-9c70-2d4dcb50bfc1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.939570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41194ebf-7155-4d4a-b18e-26015a941381 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:12.974298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc552222-cb8d-4819-b588-d13af46c6953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.975419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552222-cb8d-4819-b588-d13af46c6953 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.975837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552222-cb8d-4819-b588-d13af46c6953 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.976274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc552222-cb8d-4819-b588-d13af46c6953 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.977597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc552222-cb8d-4819-b588-d13af46c6953 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.979176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc552222-cb8d-4819-b588-d13af46c6953 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.981422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 128/266] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:12.982743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c896-cc44-40f7-b4bc-d3b0d865ac1b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.983310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c896-cc44-40f7-b4bc-d3b0d865ac1b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.983943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c896-cc44-40f7-b4bc-d3b0d865ac1b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.985599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 129/267] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:12.987991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb1db6-de03-43f4-8222-214844c82665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:12.988480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243c896-cc44-40f7-b4bc-d3b0d865ac1b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:12.988878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1db6-de03-43f4-8222-214844c82665 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.989156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1db6-de03-43f4-8222-214844c82665 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.989432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1db6-de03-43f4-8222-214844c82665 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:12.990944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb1db6-de03-43f4-8222-214844c82665 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:12.992401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb1db6-de03-43f4-8222-214844c82665 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:12.994103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 139/268] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:12.998702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:12.998702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:12.998702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.024333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:13.025751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:13.127062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243c896-cc44-40f7-b4bc-d3b0d865ac1b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:13.151426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 140/269] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:13.152516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:13.180900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:13.182482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.182966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.183595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.185244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=f0d0beaec57e4af5ba5e5146dd9d2acb) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:13.185758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:13.188468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:13.201510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 130/270] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:12 2022] GET /identity/v3/role_assignments?user.id=f0d0beaec57e4af5ba5e5146dd9d2acb&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:13.203074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:13.296737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1b833-1ef5-406e-b752-cd6e54367638 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:13.333300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173d224-bc47-4f82-adf6-efc5beb5efc7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:13.336598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:13.337339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.337818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.338296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.339318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] RBAC: Adding query filter params (role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=f0d0beaec57e4af5ba5e5146dd9d2acb) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:13.339816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:13.341471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65d755-db4d-4541-b6f1-08d1bcff86c1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:13.348152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 141/271] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:13 2022] GET /identity/v3/role_assignments?user.id=f0d0beaec57e4af5ba5e5146dd9d2acb&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:13.349898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be431c-7fcb-4c82-ba3b-8548ef75169c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.350365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be431c-7fcb-4c82-ba3b-8548ef75169c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.350817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be431c-7fcb-4c82-ba3b-8548ef75169c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.352273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 142/272] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:13.358036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.358819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.359292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.378122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:13.379607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:13.386450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:13.387263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.387541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.387804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.389175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:13.391438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:13.396267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 131/273] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:13.399458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:13.454401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 143/274] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:13.479243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:13.524030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f3f0e-0182-4e11-959b-bcd5bb35cff5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:13.559838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:13.560542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.561631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.561631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.562307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:13.563740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:13.589191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 132/275] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:13 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:13.611825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e5ccb-2f0e-47e2-b612-d0ee89286002 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:13.654465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:13.655396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.656020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.656020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.670091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:13.678676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:13.679639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:13.680486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:13.684466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 144/276] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:13.687120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:13.814919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc38d0-33e1-4423-88d8-4f511a93be87 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:13.854076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:13.854778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.855061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.855332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.856099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:13.856441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:13.857906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048dbf8-528f-487c-a234-213a67b8e507 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:13.868560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 133/277] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:13.874715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:13.877352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02237626-57a7-4a6d-9e60-4df67549b69d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.877830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02237626-57a7-4a6d-9e60-4df67549b69d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.878305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02237626-57a7-4a6d-9e60-4df67549b69d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.879827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 134/278] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:13.892269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:13.892819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:13.893337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:13.910296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:13.911840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:13.995771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 135/279] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:14.003105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b61d5-b7dc-4adf-a174-d3a65f1c6f2e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:14.018743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.049864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:14.051381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.051842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.052531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] PATH_INFO: `/v3/users/b624de3969be4778bf9ccca014d0104e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.068913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] RBAC: Authorizing `identity:get_user(user_id=b624de3969be4778bf9ccca014d0104e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:14.070660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:14.072946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 145/280] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:13 2022] GET /identity/v3/users/b624de3969be4778bf9ccca014d0104e => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:14.079106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.145458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fc119-5ad1-4a2d-8df3-bc0bb64dfea1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:14.182768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:14.183561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.184306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.184590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.198078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0d13f-682f-45e0-a7a8-5b048bc2ea32 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:14.198395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:14.207180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:14.208083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:14.208919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 136/281] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:14.215529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.237925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:14.238693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.238977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.239300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.240435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, user.id=b624de3969be4778bf9ccca014d0104e, scope.system=all) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:14.240790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:14.242227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:14.247585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 146/282] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 12 17:31:14 2022] GET /identity/v3/role_assignments?user.id=b624de3969be4778bf9ccca014d0104e&scope.system=all&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:14.325960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def7ee3-fbc3-444e-af76-cbb999f6b353 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:14.362747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:14.363517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.363696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.363948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.364738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:14.365043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:14.366724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:14.380282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 137/283] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:14.381940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.497492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c47b5-c848-4cc3-8386-9856b5b0191e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:14.534756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc9755-989d-4165-87f0-cca61023e6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:14.535633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc9755-989d-4165-87f0-cca61023e6cd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.535971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc9755-989d-4165-87f0-cca61023e6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.536276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc9755-989d-4165-87f0-cca61023e6cd None admin] PATH_INFO: `/v3/users/56238a3816404b33a4dd4f1c0cea0ec8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.537524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc9755-989d-4165-87f0-cca61023e6cd None admin] RBAC: Authorizing `identity:get_user(user_id=56238a3816404b33a4dd4f1c0cea0ec8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:14.538957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc9755-989d-4165-87f0-cca61023e6cd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:14.540644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 147/284] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:14 2022] GET /identity/v3/users/56238a3816404b33a4dd4f1c0cea0ec8 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:14.546741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.651384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b652d-b9ba-41fc-81c3-0c0d09452985 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.651731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b652d-b9ba-41fc-81c3-0c0d09452985 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.651929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b652d-b9ba-41fc-81c3-0c0d09452985 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.653539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 148/285] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:14.659507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.659794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.660101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.677581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:14.678780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:14.681410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20cf72-f81a-430a-bc32-0de75022c58a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:14.748355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:14.750477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:14.750974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:14.751461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:14.753164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:14.755445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:14.757776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 138/286] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:14 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:14.768742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.818762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 149/287] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:14.879258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:14.921933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f40f4-6045-4735-8c0a-1a93eee0465d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:15.001775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:15.002741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:15.003215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:15.003717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/56238a3816404b33a4dd4f1c0cea0ec8/roles/3867c58a744345018afcba3c9b9fb2e5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:15.005979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7b5bc85b1406083bf0c40dee773d1, user_id=56238a3816404b33a4dd4f1c0cea0ec8, role_id=3867c58a744345018afcba3c9b9fb2e5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:15.007704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:15.022904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacc59d-5d0b-42d5-8cf4-0dde8647d938 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:15.049131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 139/288] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:14 2022] PUT /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/users/56238a3816404b33a4dd4f1c0cea0ec8/roles/3867c58a744345018afcba3c9b9fb2e5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:15.069492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:15.070892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:15.071378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:15.071975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:15.088145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:15.097628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:15.098592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:15.099397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:15.102209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 150/289] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:15.108655 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:15.288661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74152a8-f68f-4b37-aaf7-6e52966d9186 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:15.338416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:15.339101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:15.339386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:15.339661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:15.340432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:15.340742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:15.342199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:15.361988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 140/290] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:15.368205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:15.554381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d49a65-dddb-4f4c-8e16-fd4ede4d692f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:15.596289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:15.597734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:15.598214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:15.598710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] PATH_INFO: `/v3/users/a68875542cac48df81d344fdc20bd3a6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:15.604986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=a68875542cac48df81d344fdc20bd3a6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:15.606716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:15.609572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 151/291] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:15 2022] GET /identity/v3/users/a68875542cac48df81d344fdc20bd3a6 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:15.617461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:15.768102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e212c59-b67e-4f4a-a6d8-e8cea782bf17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:15.815880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:15.816488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:15.816769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:15.817043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:15.894787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:15.896288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:15.993277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 141/292] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:15.998799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:16.115008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07336a-aad9-4492-9d12-4eb8ff84c5c6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:16.147910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c549d2b-d5b1-4184-93bd-40c72d59883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:16.148801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c549d2b-d5b1-4184-93bd-40c72d59883c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.149084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c549d2b-d5b1-4184-93bd-40c72d59883c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.149358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c549d2b-d5b1-4184-93bd-40c72d59883c None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:16.150536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c549d2b-d5b1-4184-93bd-40c72d59883c None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:16.151959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c549d2b-d5b1-4184-93bd-40c72d59883c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:16.153963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 152/293] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:15 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:16.159731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:16.258527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4050fa-d4df-4149-9d5e-170a3411756e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:16.293113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:16.293765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.294043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.294314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:16.295140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=a68875542cac48df81d344fdc20bd3a6) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:16.295468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:16.296993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd9275-8b27-4f14-93a7-b7cd3adf1e6c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:16.303068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 142/294] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:16 2022] GET /identity/v3/role_assignments?user.id=a68875542cac48df81d344fdc20bd3a6&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:16.885332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b8ea-27de-480b-b88e-ae85a6cf66d4 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.885520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b8ea-27de-480b-b88e-ae85a6cf66d4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.885804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b8ea-27de-480b-b88e-ae85a6cf66d4 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:16.887165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 153/295] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:16.893508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f4b9-6dbc-4280-9407-f226eca61530 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.893951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f4b9-6dbc-4280-9407-f226eca61530 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.894046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f4b9-6dbc-4280-9407-f226eca61530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:16.910775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-01e4f4b9-6dbc-4280-9407-f226eca61530 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:16.911941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01e4f4b9-6dbc-4280-9407-f226eca61530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:16.985201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 143/296] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:16.989635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6598-789e-4719-9d9b-7c4be24a2a55 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.989796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e11bc-dfcb-404f-b2f8-08f752223948 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.990324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e11bc-dfcb-404f-b2f8-08f752223948 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.990610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e11bc-dfcb-404f-b2f8-08f752223948 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:16.990939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6598-789e-4719-9d9b-7c4be24a2a55 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.991406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6598-789e-4719-9d9b-7c4be24a2a55 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:16.992009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 144/297] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:16.993362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 154/298] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:16.998306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488642c-5b26-4735-819a-c69a284143fa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:16.998615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488642c-5b26-4735-819a-c69a284143fa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:16.998924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488642c-5b26-4735-819a-c69a284143fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.001193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.002031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.002499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.018848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f488642c-5b26-4735-819a-c69a284143fa None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:17.020028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f488642c-5b26-4735-819a-c69a284143fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:17.021673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:17.023360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:17.092217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 145/299] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:17.092217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488642c-5b26-4735-819a-c69a284143fa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.095503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 155/300] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:17.110659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.183734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488642c-5b26-4735-819a-c69a284143fa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.209477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14deaaff-4e04-401d-9ab0-99bdee85f37d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.216151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.216974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.217255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.218061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.227666 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:17.235451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:17.236402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:17.237178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 146/301] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:17.237833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.248092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.250030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.250317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.250595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.260878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:17.269196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:17.270291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:17.271207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 156/302] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:17.275745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.340711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6267c1-680a-4e3d-ab7d-cd4b2da286e2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.372062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e7420-4328-4115-b04f-7d707a01b02c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.373000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.374440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.374726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.374992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.376909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.378383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.380022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 147/303] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:17.380891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.412973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.413386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.413675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.413939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.414687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:17.414987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.416370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.425009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 157/304] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:17.425776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.497906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae14aaa-1007-4211-a302-d2ef9a55547b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.524959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52490e-9e34-4a8c-a557-0d674c44c4fd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.532868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.533539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.533821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.534305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.534832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:17.535135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.536526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.546159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 148/305] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:17.546159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.563587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.564300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.564587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.564863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] PATH_INFO: `/v3/groups` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.565552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.566997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.581280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 158/306] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 12 17:31:17 2022] POST /identity/v3/groups => generated 229 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:17.584242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.674058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b3fa0-8af5-434e-a0e0-c0c23d7818ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.715921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.716801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.717079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.718076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] PATH_INFO: `/v3/users/28aa91fdc5c44d438c88a2b5c5e79aaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.726782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9683636-a1fc-4ca1-8754-e307bf441b4f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.739811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] RBAC: Authorizing `identity:get_user(user_id=28aa91fdc5c44d438c88a2b5c5e79aaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.742541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.745759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 149/307] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/users/28aa91fdc5c44d438c88a2b5c5e79aaf => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:17.752307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.768497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.769122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.769396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.769730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] PATH_INFO: `/v3/users/b624de3969be4778bf9ccca014d0104e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.771028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] RBAC: Authorizing `identity:get_user(user_id=b624de3969be4778bf9ccca014d0104e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.772609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.774640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 159/308] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/users/b624de3969be4778bf9ccca014d0104e => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:17.782233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:17.870002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0b242-f2e7-4545-befd-5bee7bf78709 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.904087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e83a5-b429-40bf-8762-ed75327eb25e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:17.904468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.906025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.906025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.906430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.907266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] RBAC: Adding query filter params (role.id=3867c58a744345018afcba3c9b9fb2e5, user.id=28aa91fdc5c44d438c88a2b5c5e79aaf, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:17.907573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.909025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f983f-83de-4b6e-82ec-efc5baa7105c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.914059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 150/309] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 12 17:31:17 2022] GET /identity/v3/role_assignments?user.id=28aa91fdc5c44d438c88a2b5c5e79aaf&scope.system=all&role.id=3867c58a744345018afcba3c9b9fb2e5 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:17.916609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e45f7b-8bda-4d76-af58-65cfe2f635a6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.916897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e45f7b-8bda-4d76-af58-65cfe2f635a6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.917176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e45f7b-8bda-4d76-af58-65cfe2f635a6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.918630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 151/310] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:17.930531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.930531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.930781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.948923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:17.949942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:17.950336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:17.950799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:17.951252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] PATH_INFO: `/v3/system/users/b624de3969be4778bf9ccca014d0104e/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:17.951379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:17.954011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b624de3969be4778bf9ccca014d0104e, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:17.956959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:17.969214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 160/311] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 12 17:31:17 2022] PUT /identity/v3/system/users/b624de3969be4778bf9ccca014d0104e/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:18.040774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 152/312] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:18.066039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:18.183373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6404f8e-870b-4046-a17c-68f303e657d6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:18.218262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:18.218805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:18.219087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:18.219360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:18.229995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:18.238711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:18.239596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:18.240675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 161/313] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:18.246107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:18.347032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b097cb3-70cc-4699-8522-34f96f076b37 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:18.381548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:18.382232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:18.382514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:18.382784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:18.383538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:18.383840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:18.385241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:18.394792 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 153/314] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:18.400473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:18.504287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88df5ea-6bfc-4aa9-83ef-d4e633d1a3ed None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:18.539924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:18.541172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:18.541656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:18.542158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] PATH_INFO: `/v3/users/56238a3816404b33a4dd4f1c0cea0ec8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:18.543552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] RBAC: Authorizing `identity:get_user(user_id=56238a3816404b33a4dd4f1c0cea0ec8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:18.545229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:18.548056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 162/315] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:18 2022] GET /identity/v3/users/56238a3816404b33a4dd4f1c0cea0ec8 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:18.559066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:18.676139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bc447-1705-48f8-bd15-48ca2cefc03c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:18.709667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:18.710324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:18.710608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:18.710880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:18.777960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:18.779513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:18.884340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 154/316] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:18.889362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:18.990710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b874-6b5b-40c0-ae2a-f500a793188a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:19.023536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71653e0-0368-40cf-9f83-a95bfd7b2689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:19.024419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71653e0-0368-40cf-9f83-a95bfd7b2689 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:19.024702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71653e0-0368-40cf-9f83-a95bfd7b2689 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:19.024977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71653e0-0368-40cf-9f83-a95bfd7b2689 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:19.026160 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71653e0-0368-40cf-9f83-a95bfd7b2689 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:19.027592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71653e0-0368-40cf-9f83-a95bfd7b2689 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:19.029600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 163/317] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:18 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:19.035523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:19.133615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b48d63-fa35-47de-98b3-a9bff48a48a2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:19.167542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:19.168227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:19.168511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:19.168788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:19.169620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] RBAC: Adding query filter params (role.id=3867c58a744345018afcba3c9b9fb2e5, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1, user.id=56238a3816404b33a4dd4f1c0cea0ec8) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:19.169927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:19.172152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a642f-6b38-46a3-806d-85713cce5bce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:19.178138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 155/318] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:19 2022] GET /identity/v3/role_assignments?user.id=56238a3816404b33a4dd4f1c0cea0ec8&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=3867c58a744345018afcba3c9b9fb2e5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:19.929450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cc7d4-0d4d-47fe-a02f-47318a1a1163 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:19.929991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cc7d4-0d4d-47fe-a02f-47318a1a1163 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:19.929991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cc7d4-0d4d-47fe-a02f-47318a1a1163 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:19.931516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 164/319] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:19.936615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426bb3a-766f-4757-89e0-5e1d6297bade None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:19.936903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426bb3a-766f-4757-89e0-5e1d6297bade None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:19.937252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426bb3a-766f-4757-89e0-5e1d6297bade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:19.954463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d426bb3a-766f-4757-89e0-5e1d6297bade None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:19.955657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d426bb3a-766f-4757-89e0-5e1d6297bade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:20.023123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 156/320] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:20.046254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cc7d4-0d4d-47fe-a02f-47318a1a1163 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.144703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cc7d4-0d4d-47fe-a02f-47318a1a1163 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.177036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.177819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.178101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.178372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.179563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:20.180977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:20.182778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 165/321] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:20.188827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66639453-0937-4c12-a87a-89be98d540ef None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.189289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66639453-0937-4c12-a87a-89be98d540ef None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.189398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66639453-0937-4c12-a87a-89be98d540ef None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.190695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 157/322] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:20.191745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.197061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.197434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.197780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.214444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:20.215610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:20.283528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 158/323] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:20.293875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b87e-fdef-4831-8662-9c5eeed026f5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.305836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.327169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b5f93-282f-47f9-8dc2-ac67100c558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.327834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b5f93-282f-47f9-8dc2-ac67100c558b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.328115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b5f93-282f-47f9-8dc2-ac67100c558b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.328420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b5f93-282f-47f9-8dc2-ac67100c558b None admin] PATH_INFO: `/v3/groups` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.329110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b5f93-282f-47f9-8dc2-ac67100c558b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:20.330528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b5f93-282f-47f9-8dc2-ac67100c558b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:20.352200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 166/324] 10.222.0.73 () {60 vars in 1134 bytes} [Thu May 12 17:31:20 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:20.419177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36d6ed-3a20-46b6-83b7-783e5cf2e67f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.448911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556a032-0c24-42da-8650-45972f26739d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.449106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556a032-0c24-42da-8650-45972f26739d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.449377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556a032-0c24-42da-8650-45972f26739d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.454825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 167/325] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:20.456876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.457166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.457458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.458568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.459366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.459702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.459918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.471672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:20.474467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:20.475693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:20.479554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:20.480507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:20.481437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 159/326] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:20.487633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.557770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 168/327] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:20.582910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.605677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bdc9c-c721-45e8-a659-f957e57930f9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.653945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.654617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.655451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.655721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.656706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:20.657010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:20.658382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:20.667495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 160/328] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:20.674019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.708485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407fd0a-0ded-474a-a7f0-bfd1a96e200d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.750767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.753223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.753223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.753223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.765865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:20.773528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:20.774412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:20.775208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:20.779564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 169/329] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:20.781441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.792761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f413-31d1-4fe8-8f0b-5dc68dd69f53 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.829737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.830610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.830953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.831233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] PATH_INFO: `/v3/users/28aa91fdc5c44d438c88a2b5c5e79aaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.832527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] RBAC: Authorizing `identity:get_user(user_id=28aa91fdc5c44d438c88a2b5c5e79aaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:20.833983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:20.835757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 161/330] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/users/28aa91fdc5c44d438c88a2b5c5e79aaf => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:20.841285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.885289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859048b-fe94-4e27-bcbc-2b0b9b8f7ce6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.920074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.921405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.921688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.921953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.922703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:20.923004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:20.924411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:20.936769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 170/331] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:20.939265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:20.948078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa36f23-f1f6-4828-89e1-e2fd6cf37a6f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:20.984263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:20.987059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:20.987059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:20.987059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] PATH_INFO: `/v3/system/users/28aa91fdc5c44d438c88a2b5c5e79aaf/roles/3867c58a744345018afcba3c9b9fb2e5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:20.992261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=28aa91fdc5c44d438c88a2b5c5e79aaf, role_id=3867c58a744345018afcba3c9b9fb2e5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:20.992261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:21.000856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 162/332] 10.222.0.73 () {58 vars in 1334 bytes} [Thu May 12 17:31:20 2022] PUT /identity/v3/system/users/28aa91fdc5c44d438c88a2b5c5e79aaf/roles/3867c58a744345018afcba3c9b9fb2e5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:31:21.049286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b443-9537-4f99-a595-c1633c01ac82 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:21.087566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0c644-07be-4336-aa50-8979a3823826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:21.088529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c644-07be-4336-aa50-8979a3823826 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:21.088808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c644-07be-4336-aa50-8979a3823826 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:21.089079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c644-07be-4336-aa50-8979a3823826 None admin] PATH_INFO: `/v3/users/b624de3969be4778bf9ccca014d0104e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:21.090332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0c644-07be-4336-aa50-8979a3823826 None admin] RBAC: Authorizing `identity:get_user(user_id=b624de3969be4778bf9ccca014d0104e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:21.091775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0c644-07be-4336-aa50-8979a3823826 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:21.098700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 171/333] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:20 2022] GET /identity/v3/users/b624de3969be4778bf9ccca014d0104e => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:21.100344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:21.210713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788adf49-55d0-4fc1-9cff-596038a0afb4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:21.250424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:21.251003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:21.251279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:21.251548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:21.252465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] RBAC: Adding query filter params (role.id=039b03887b8b49dfab13afaf1d54bfc0, user.id=b624de3969be4778bf9ccca014d0104e, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:21.252786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:21.254195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06c906-db1e-47b8-a14b-5c8f886ac87e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:21.266141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 163/334] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 12 17:31:21 2022] GET /identity/v3/role_assignments?user.id=b624de3969be4778bf9ccca014d0104e&scope.system=all&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:23.348749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4867-65e4-4c7c-aa99-bb9900ae0f25 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:23.349322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4867-65e4-4c7c-aa99-bb9900ae0f25 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:23.349322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4867-65e4-4c7c-aa99-bb9900ae0f25 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:23.350627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 172/335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:23.366387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:23.367006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:23.367006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:23.377780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626872ad-08ec-4271-b786-46dd192034d7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:23.377780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626872ad-08ec-4271-b786-46dd192034d7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:23.377780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626872ad-08ec-4271-b786-46dd192034d7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:23.378615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 173/336] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:23.382700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:23.383203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:23.383706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:23.388954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:23.390586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:23.417150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:23.418690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:23.474388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 164/337] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:23.534707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:23.536618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 174/338] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:23.568187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:23.704015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c35e6-50e5-4545-a884-6ae8252976ad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:23.755907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:23.757159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:23.757662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:23.758169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:23.773679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:23.786916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:23.787876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:23.788749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:23.791269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 165/339] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:23.801150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:23.808314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f3837-486e-4b9e-a745-8a1cdbd69a59 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:23.886451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:23.887183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:23.887467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:23.887774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:23.910197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:23.926234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:23.927189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:23.927976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 175/340] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:23.934623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.003003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c787ad5-0ace-4005-9f93-d0141e97f162 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.044132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc8297-d7fc-418a-84cd-c5ea6e456ab6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.081723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.082400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.082675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.082953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.083713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:24.084018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:24.085437 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58c5fa-b178-4780-8e1a-88d9c2420c9e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:24.094715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 176/341] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:24.096028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616573c3-ea8e-4279-8643-90eee3170efa None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.096422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616573c3-ea8e-4279-8643-90eee3170efa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.096703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616573c3-ea8e-4279-8643-90eee3170efa None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.097922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 177/342] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:24.099599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.100293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.100574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.100843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.101590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:24.101888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:24.103254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b912155-7114-4abd-a0c3-3833048a870d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:24.112234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616573c3-ea8e-4279-8643-90eee3170efa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.120299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 166/343] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:24.121565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.121848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.122120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.153817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:24.155059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:24.281077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616573c3-ea8e-4279-8643-90eee3170efa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.290746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 167/344] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:24.291675 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.355332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.356007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.356291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.356565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.476901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:24.478261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:24.490117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932adf9-5f4f-4858-8fb0-02387de490d3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.564981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.565991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.566499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.566986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.585595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 178/345] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:24.586361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.651450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:24.653994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:24.698757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33710ba1-7b08-4341-a535-8b6b17877a7d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.737124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182babd-fc96-49e0-bfab-3f84e311a4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.738293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182babd-fc96-49e0-bfab-3f84e311a4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.738293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182babd-fc96-49e0-bfab-3f84e311a4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.738489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182babd-fc96-49e0-bfab-3f84e311a4a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.751488 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182babd-fc96-49e0-bfab-3f84e311a4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7182babd-fc96-49e0-bfab-3f84e311a4a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:24.759288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:24.760303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:24.761112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 179/346] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:24.767806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dad29-a488-4c1a-92f9-659478743106 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.768192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dad29-a488-4c1a-92f9-659478743106 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.768655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dad29-a488-4c1a-92f9-659478743106 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.770151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 180/347] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:24.770974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dad29-a488-4c1a-92f9-659478743106 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.776467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 168/348] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:24.777535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.887460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dad29-a488-4c1a-92f9-659478743106 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.893365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc54f18-0541-4b0e-8dcf-86881d5041ec None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:24.934268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180e075-e37f-45af-b39a-0db126f48486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.935724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180e075-e37f-45af-b39a-0db126f48486 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.936207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180e075-e37f-45af-b39a-0db126f48486 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.936728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180e075-e37f-45af-b39a-0db126f48486 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.936972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:24.937762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.938142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.938511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.938836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180e075-e37f-45af-b39a-0db126f48486 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:24.939456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:24.939859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:24.940832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180e075-e37f-45af-b39a-0db126f48486 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:24.941461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:24.943204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 181/349] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:24 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:24.944588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41d948-5121-4255-94da-f91ee829f96a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:24.945075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41d948-5121-4255-94da-f91ee829f96a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:24.945522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41d948-5121-4255-94da-f91ee829f96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:24.954286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 169/350] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:24.955272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:24.969208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-de41d948-5121-4255-94da-f91ee829f96a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:24.970721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de41d948-5121-4255-94da-f91ee829f96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:25.047215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 182/351] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:25.048383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41d948-5121-4255-94da-f91ee829f96a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.072326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ddc5-0a89-4dd7-b59b-8ad83972f9f9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.107192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.108456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.108837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.109214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.110536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.112082 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.114246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 170/352] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:24 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.115125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.158396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41d948-5121-4255-94da-f91ee829f96a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.207880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.209015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.209440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.209866 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.216808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.218831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.220846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 183/353] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.221326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.241898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953317-05b3-4574-aebc-7a8cdb277b69 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.282302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1d5e4-61dc-411f-8daf-a0c2e3f38281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.283880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1d5e4-61dc-411f-8daf-a0c2e3f38281 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.284495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1d5e4-61dc-411f-8daf-a0c2e3f38281 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.285048 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1d5e4-61dc-411f-8daf-a0c2e3f38281 None admin] PATH_INFO: `/v3/users/28aa91fdc5c44d438c88a2b5c5e79aaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.287115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1d5e4-61dc-411f-8daf-a0c2e3f38281 None admin] RBAC: Authorizing `identity:get_user(user_id=28aa91fdc5c44d438c88a2b5c5e79aaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.289285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1d5e4-61dc-411f-8daf-a0c2e3f38281 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.292010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 171/354] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/users/28aa91fdc5c44d438c88a2b5c5e79aaf => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.293004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f253f-19e7-4806-8afd-2a8083cc8892 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.293263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f253f-19e7-4806-8afd-2a8083cc8892 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.296772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f253f-19e7-4806-8afd-2a8083cc8892 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.296772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 172/355] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:25.296772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f253f-19e7-4806-8afd-2a8083cc8892 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.345595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eb348-7cc7-4b77-9da9-4e5257b9334c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.383277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508a52-c015-4e13-8def-9cce2e11e887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.384269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508a52-c015-4e13-8def-9cce2e11e887 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.384525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508a52-c015-4e13-8def-9cce2e11e887 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.384796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508a52-c015-4e13-8def-9cce2e11e887 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.396927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508a52-c015-4e13-8def-9cce2e11e887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f7508a52-c015-4e13-8def-9cce2e11e887 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:25.405200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:25.406097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:25.406886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 184/356] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:25.409717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62277825-d513-40e0-befb-0eba35e09e8f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.410009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62277825-d513-40e0-befb-0eba35e09e8f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.410295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62277825-d513-40e0-befb-0eba35e09e8f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.411773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 185/357] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:25.412502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62277825-d513-40e0-befb-0eba35e09e8f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.425363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f253f-19e7-4806-8afd-2a8083cc8892 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.463439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.464803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.464803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.465184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.467839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.469790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.471466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 173/358] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.472266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.526400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62277825-d513-40e0-befb-0eba35e09e8f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.565837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.565837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.565837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.566200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.566921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=8770b556c0c24d1b85b244d3070b0330) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:25.567229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.568688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ae696-5f75-437a-8474-e486049d34e0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.574776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 186/359] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.575996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8526-283d-47d4-9de0-6ade3cacb747 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.576375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8526-283d-47d4-9de0-6ade3cacb747 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.576767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff8526-283d-47d4-9de0-6ade3cacb747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.595960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-adff8526-283d-47d4-9de0-6ade3cacb747 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:25.597573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-adff8526-283d-47d4-9de0-6ade3cacb747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:25.603525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56947d88-b015-41bc-8852-a0e69746f1aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.662569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.663524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.663818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.664092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.665160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] RBAC: Adding query filter params (role.id=3867c58a744345018afcba3c9b9fb2e5, user.id=28aa91fdc5c44d438c88a2b5c5e79aaf, scope.system=all) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:25.665875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.667293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.686155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 174/360] 10.222.0.73 () {56 vars in 1308 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/role_assignments?user.id=28aa91fdc5c44d438c88a2b5c5e79aaf&scope.system=all&role.id=3867c58a744345018afcba3c9b9fb2e5 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.687108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.695646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 187/361] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:25.697268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.697782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.698243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.729257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:25.730883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:25.809420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 188/362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:25.810518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.826083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8096b-9e21-4bcb-baac-536e511b3358 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.864432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.865465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.865846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.866255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.867095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:25.867639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.868970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.879090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 175/363] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.880213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:25.923683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c759d6a-fe46-4c1b-9a47-a44fcebf2d4a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:25.968304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:25.969351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:25.969860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:25.970339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:25.971386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=8770b556c0c24d1b85b244d3070b0330) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:25.971887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:25.973552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:25.985969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 189/364] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:25.986822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.007611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a82f4-7bf3-49a0-b5ad-b18466c5ec90 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.047487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.048349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.048630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.048900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.060734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:26.069296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:26.070251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:26.071042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 176/365] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:26.071679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.104551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60743a61-85f4-422f-afef-4630f95c3b92 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.143184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.143907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.144229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.144512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.159074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:26.166890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:26.169078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 190/366] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:26.171088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.185065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9212-25a6-4eff-bcd9-dab8542331ad None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.223191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.223851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.224131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.224437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.285825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875623c-49b9-4f28-b08f-51fd1ede5b63 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.299987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:26.301508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:26.325605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.327159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.328028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.328528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.329907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:26.330446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:26.332898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:26.349304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 191/367] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:26.350589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.468127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41cc5-752d-428b-b176-e821b34a890b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.476559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 177/368] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:26.477838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.505162 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.505841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.506123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.506400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.507151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:26.507458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:26.508863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:26.517553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 192/369] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:26.518355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.628098 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6adf-5be9-4259-829c-894e30a04597 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.658198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d4fda-fb7f-458f-86be-7a79d8995c9b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.662015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.662889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.663166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.663438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.664695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:26.666143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:26.667831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 193/370] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:26.668607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:26.726021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.726957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.727434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.727897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.773448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50b29b-2939-4fbc-9ff8-d4d6374739a9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:26.809152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:26.809799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:26.810082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:26.810354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:26.848360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:26.850787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:26.888330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:26.890883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:26.987426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 178/371] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:26.988366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.006346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 194/372] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:27.007198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.105772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705794b3-0113-432e-aa91-3c96f685dbb4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.129781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8872d22-a605-4c3a-9f4d-f437cc1dd672 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.155955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.157365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.157365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.157365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.160621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.162089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.163766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 179/373] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:26 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.165261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.178763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.179617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.179907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.180208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.181434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.182870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.184915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 195/374] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.185704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.277610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd04f4-c4e2-4804-bcd2-aafefa3fcb6e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.292087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589b10-67f4-4952-bd3e-d13fd6346a9f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.312185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.313343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.313762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.314189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] PATH_INFO: `/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.315518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] RBAC: Authorizing `identity:get_project(project_id=ccf7092b8d0b4d6a9522dc9ba6d91833)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.317119 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.318967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 180/375] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.319858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.335029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.335685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.335967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.336290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.337121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:27.337433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.338878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.345461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 196/376] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.346376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.426394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac2ed7-4f1e-44a7-be17-6ca2876815f6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.463421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.464633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.465079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.465490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.466834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.469092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.471202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 181/377] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.477039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.491288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea0688-0f2e-4f98-b341-c503164ed958 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.526774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.527713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.527940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.528262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] PATH_INFO: `/v3/groups/c55b414aa0404ad182715fe53bf5fe41` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.534731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] RBAC: Authorizing `identity:get_group(group_id=c55b414aa0404ad182715fe53bf5fe41)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.536247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.538200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 197/378] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/groups/c55b414aa0404ad182715fe53bf5fe41 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.544132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:27.594490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95be79-be34-4480-84d2-b5bfd22140aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.638699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.640402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.641025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.641438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.642412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:27.642881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.644558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05b4b-7a1f-4976-beaf-8488166e19bb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.651035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 182/379] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:27.657318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f95220-abf8-4c72-8388-951b288ceb42 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:27.698364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:27.699037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:27.699323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:27.699616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:27.700483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] RBAC: Adding query filter params (group.id=c55b414aa0404ad182715fe53bf5fe41, role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ccf7092b8d0b4d6a9522dc9ba6d91833) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:27.700798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:27.703233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28f03-98ac-45f0-872a-521ecdb1af18 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:27.710083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 198/380] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:27 2022] GET /identity/v3/role_assignments?group.id=c55b414aa0404ad182715fe53bf5fe41&scope.project.id=ccf7092b8d0b4d6a9522dc9ba6d91833&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:28.530783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eedaa-10ae-4b7b-882a-8314c65aa1e6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:28.531809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eedaa-10ae-4b7b-882a-8314c65aa1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:28.532323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eedaa-10ae-4b7b-882a-8314c65aa1e6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:28.534196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 183/381] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:28.539548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1161177-5e2b-4054-8b70-a112e7787cda None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:28.540403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1161177-5e2b-4054-8b70-a112e7787cda None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:28.540876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1161177-5e2b-4054-8b70-a112e7787cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:28.562784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d1161177-5e2b-4054-8b70-a112e7787cda None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:28.564403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d1161177-5e2b-4054-8b70-a112e7787cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:28.648190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 199/382] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:28.685674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eedaa-10ae-4b7b-882a-8314c65aa1e6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:28.804665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eedaa-10ae-4b7b-882a-8314c65aa1e6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:28.816670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed0d19-a7ad-4366-a46b-9f8119bfee6f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:28.816963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed0d19-a7ad-4366-a46b-9f8119bfee6f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:28.817250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed0d19-a7ad-4366-a46b-9f8119bfee6f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:28.819597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 200/383] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:28.825524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:28.825811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:28.826093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:28.843758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:28.845049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:28.845529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:28.846056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:28.857181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:28.858257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:28.858930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:28.866922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:28.867807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:28.868622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:28.871473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 184/384] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:28.881623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:28.989926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d16f1-1b39-4d3f-b518-ddf9f338ae98 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.024708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.025861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.026390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.026493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 201/385] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:29.026951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.027898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:29.028420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.030998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.040207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 185/386] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:29.045717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.058523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.161332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da8e21-9e5b-4dde-ab82-08b7b57d3a94 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.195225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.196406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.196888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.197373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.209013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:29.218221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:29.219186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:29.219976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:29.222758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 186/387] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:29.225237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85288f2-0051-49a7-8e60-40c3a6c8127c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.228822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.275534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.276426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.276712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.276990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.281203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.282670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.284761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 202/388] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:29 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:29.289883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.336512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430565-2992-4cd8-aef8-228b098e60b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.371903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.373024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.374083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.374555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.375518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:29.376106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.377773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.387825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 187/389] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:29.394945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.427165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e165-165f-4964-a796-ce2afb3029fb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.472890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.473758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.474040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.474315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.475566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.477011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.478987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 203/390] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:29 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:29.484767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.521766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ea5b8-892a-4d82-bcf8-31b8fe2bcfae None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.559007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.560819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.561971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.562543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.564030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.565745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.567919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 188/391] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:29 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:29.573690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.621845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2c752-2cb0-418d-a01a-94f73ff33f1d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.664437 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.665319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.665595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.665866 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.667818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8770b556c0c24d1b85b244d3070b0330, group_id=3e1d433709624bfda8483d85ffe1b0f5, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.669379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.694761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31891428-ce70-49b2-b400-18c72fde3f1f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.698060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 204/392] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 12 17:31:29 2022] PUT /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:29.734475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c626a7-954e-4492-a2cc-02a4926654d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.735710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c626a7-954e-4492-a2cc-02a4926654d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.736235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c626a7-954e-4492-a2cc-02a4926654d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.736705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c626a7-954e-4492-a2cc-02a4926654d5 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.738329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c626a7-954e-4492-a2cc-02a4926654d5 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.740101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c626a7-954e-4492-a2cc-02a4926654d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.742867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 189/393] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:29 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:29.749635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:29.856622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929bea2-4dbf-4ea4-a792-6f9fdad419d8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:29.901538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352c433-be0b-4ae1-bfff-bf298c263918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:29.902787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c433-be0b-4ae1-bfff-bf298c263918 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:29.903264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c433-be0b-4ae1-bfff-bf298c263918 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:29.903750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c433-be0b-4ae1-bfff-bf298c263918 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:29.909525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352c433-be0b-4ae1-bfff-bf298c263918 None admin] RBAC: Authorizing `identity:create_grant(project_id=8770b556c0c24d1b85b244d3070b0330, group_id=3e1d433709624bfda8483d85ffe1b0f5, role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:29.911348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352c433-be0b-4ae1-bfff-bf298c263918 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:29.939179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 205/394] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 12 17:31:29 2022] PUT /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:30.523071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9384319-acb9-465c-bbe1-b410616c0afa None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:30.523740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9384319-acb9-465c-bbe1-b410616c0afa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:30.524269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9384319-acb9-465c-bbe1-b410616c0afa None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:30.526097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 190/395] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:30.533211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:30.533745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:30.534241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:30.552443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:30.553632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:30.647169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 206/396] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:30.689330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9384319-acb9-465c-bbe1-b410616c0afa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:30.814908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9384319-acb9-465c-bbe1-b410616c0afa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:30.850049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053790-87be-42db-b6cb-275136cdf8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:30.851366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053790-87be-42db-b6cb-275136cdf8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:30.851842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053790-87be-42db-b6cb-275136cdf8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:30.852362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053790-87be-42db-b6cb-275136cdf8a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:30.863973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053790-87be-42db-b6cb-275136cdf8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-04053790-87be-42db-b6cb-275136cdf8a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:30.872504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:30.873930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:30.874702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:30.877302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 191/397] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:30.883560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:30.902707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7530b9-f451-4baf-bd29-31c4fbd7e02f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:30.903295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7530b9-f451-4baf-bd29-31c4fbd7e02f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:30.903786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7530b9-f451-4baf-bd29-31c4fbd7e02f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:30.906391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 192/398] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:30.911846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8ef0-cfd8-4d42-b045-3f9458f0a4ea None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:30.912454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8ef0-cfd8-4d42-b045-3f9458f0a4ea None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:30.912949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa8ef0-cfd8-4d42-b045-3f9458f0a4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:30.935071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7bfa8ef0-cfd8-4d42-b045-3f9458f0a4ea None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:30.936569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7bfa8ef0-cfd8-4d42-b045-3f9458f0a4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:31.013585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbd543-b4f4-4def-8789-f1875a9310cd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.040525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 193/399] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:31.042012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d2dae-29de-45e0-8f19-b77aab8ea79c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.042304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d2dae-29de-45e0-8f19-b77aab8ea79c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.042579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d2dae-29de-45e0-8f19-b77aab8ea79c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.044132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 194/400] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:31.049657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.050011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.050213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.054355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.055262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.055738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.056341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.057373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] RBAC: Adding query filter params (name=member) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:31.057890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.059530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.070174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 207/401] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:31.075398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.082291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:31.083542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:31.167694 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 195/402] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:31.176583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.203330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d03f24-94c7-4a96-9fc7-ad8d67f0d466 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.237698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.238730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.239014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.239285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.240752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.242184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.244079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 208/403] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:31.244878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.296461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d7ffc-ca60-44c1-85f8-1197fba4359a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.333018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.334230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.334701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.335160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.350769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:31.359121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:31.360030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:31.361164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:31.363397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 196/404] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:31.363503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda4b58-1959-4b45-bb7f-af994c808fe5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.364775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.407057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.408389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.408672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.408944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.425648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:31.435630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:31.436979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:31.437974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 209/405] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:31.438771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.490702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3eb810-d3f2-4a62-bf4c-a4df7ebbb2e5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.531826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.533311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.533605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.533884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.535532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.537270 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.539615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 197/406] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:31.540329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.556569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335844f-51e6-49db-83c9-baedb8b9a0b2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.604115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.605164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.605451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.605726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.606786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:31.607092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.609014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.621637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 210/407] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:31.627597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.656430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70454837-15f9-462d-815c-3ca94bbb9418 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.690112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.691520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.692002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.692494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.693432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:31.693971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.695519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.704855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 198/408] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:31.705973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.729607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541c9b0-84d5-4916-9cb6-dd6cc3266ccd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.766143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.767031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.767396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.767798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.770490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7b5bc85b1406083bf0c40dee773d1, group_id=3e1d433709624bfda8483d85ffe1b0f5, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.772088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.795929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 211/409] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 12 17:31:31 2022] PUT /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:31.796962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.814353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cb8e8-31e3-454b-97d4-51d22180b855 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:31.852320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:31.853620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:31.854254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:31.854739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:31.856186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:31.858419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:31.860704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 199/410] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:31.867145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:31.942095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb83a7-9908-4e7a-a4d7-bd8204ecbfed None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:32.011511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:32.012915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.013428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.014520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] PATH_INFO: `/v3/groups/c55b414aa0404ad182715fe53bf5fe41` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.016388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] RBAC: Authorizing `identity:get_group(group_id=c55b414aa0404ad182715fe53bf5fe41)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:32.019303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:32.022962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 212/411] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/groups/c55b414aa0404ad182715fe53bf5fe41 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:32.028709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:32.060060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e7c99-c046-456d-b0c4-32a99a87df98 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:32.103353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:32.104539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.105070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.105541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.106957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:32.108675 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:32.110891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 200/412] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:31 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:32.117039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:32.140198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26c0f-6724-4384-9697-e6c31b164e56 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:32.178820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:32.179767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.180118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.180530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] PATH_INFO: `/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.182030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] RBAC: Authorizing `identity:get_project(project_id=ccf7092b8d0b4d6a9522dc9ba6d91833)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:32.183570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:32.185671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 213/413] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:32 2022] GET /identity/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:32.192445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:32.268308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb8458-ace1-43ea-a5c2-a1d7b6608042 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:32.336877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782e8df-98f2-4412-8822-c0bbf8d3bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:32.338195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782e8df-98f2-4412-8822-c0bbf8d3bfad None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.338680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782e8df-98f2-4412-8822-c0bbf8d3bfad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.339189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782e8df-98f2-4412-8822-c0bbf8d3bfad None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.345284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782e8df-98f2-4412-8822-c0bbf8d3bfad None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7b5bc85b1406083bf0c40dee773d1, group_id=3e1d433709624bfda8483d85ffe1b0f5, role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:32.347116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782e8df-98f2-4412-8822-c0bbf8d3bfad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:32.368896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603d33-c549-416b-8eb6-39a34a6a7a69 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:32.381541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 201/414] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 12 17:31:32 2022] PUT /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1/groups/3e1d433709624bfda8483d85ffe1b0f5/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:31:32.419960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747baccc-2fef-4221-8723-e0a4aae7e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:32.421869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747baccc-2fef-4221-8723-e0a4aae7e90d None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.422373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747baccc-2fef-4221-8723-e0a4aae7e90d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.422848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747baccc-2fef-4221-8723-e0a4aae7e90d None admin] PATH_INFO: `/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833/groups/c55b414aa0404ad182715fe53bf5fe41/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.429947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747baccc-2fef-4221-8723-e0a4aae7e90d None admin] RBAC: Authorizing `identity:create_grant(project_id=ccf7092b8d0b4d6a9522dc9ba6d91833, group_id=c55b414aa0404ad182715fe53bf5fe41, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:32.431794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747baccc-2fef-4221-8723-e0a4aae7e90d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:32.460809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 214/415] 10.222.0.73 () {58 vars in 1442 bytes} [Thu May 12 17:31:32 2022] PUT /identity/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833/groups/c55b414aa0404ad182715fe53bf5fe41/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:32.536581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dadc7-24f4-4e7f-949f-b889265ad7f0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.537241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dadc7-24f4-4e7f-949f-b889265ad7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.537717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dadc7-24f4-4e7f-949f-b889265ad7f0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.539669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 202/416] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:32.545973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6260de-f685-44c8-a3fc-b121ce596ba4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.546502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6260de-f685-44c8-a3fc-b121ce596ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.546870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6260de-f685-44c8-a3fc-b121ce596ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.574224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5f6260de-f685-44c8-a3fc-b121ce596ba4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:32.575492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5f6260de-f685-44c8-a3fc-b121ce596ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:32.664736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 215/417] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:32.689541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dadc7-24f4-4e7f-949f-b889265ad7f0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:32.727703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2344b8-70a4-43c6-9e8f-588696921051 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.727703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2344b8-70a4-43c6-9e8f-588696921051 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.727968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2344b8-70a4-43c6-9e8f-588696921051 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.729416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 216/418] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:32.738571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.739165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.739165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.762733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:32.763955 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:32.851390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dadc7-24f4-4e7f-949f-b889265ad7f0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:32.881726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 217/419] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:32.900911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:32.901722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:32.902004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:32.902274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:32.905203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:32.921696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:32.929581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:32.930462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:32.931275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:32.934880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 203/420] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:32.941106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:33.020750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa8d4-b2fa-4b94-9fb9-f6d4a32bac0c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:33.064813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:33.065998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:33.066480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:33.066952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:33.091329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:33.100236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:33.101115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:33.101910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:33.114694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 218/421] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:33.122011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:33.141625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9b1b-a63f-424b-8d43-a79b5754f1ab None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:33.181933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:33.184265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:33.184754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:33.186399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:33.188076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:33.190802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:33.192404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:33.211369 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 204/422] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:33.216731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:33.303566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7448e-9d48-460a-b10b-58404c6d8c2f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:33.345276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:33.347006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:33.348820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:33.349306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:33.350304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:33.352619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:33.354305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:33.376070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 219/423] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:33.381508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:33.462602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c54de8-5f2d-4eaf-99fe-0348309516f7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:33.541308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08bc8e-aa00-452a-a1fc-5039e75a0a63 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:33.558090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:33.558737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:33.559015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:33.559287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:33.582505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:33.583491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:33.583987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:33.585876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:33.653074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:33.655318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:33.762262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 205/424] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:33.770146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:33.770771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:33.772234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:33.878463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23e33a-c3f0-4a89-870e-688e674c8e6d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:33.891989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 220/425] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 12 17:31:33.897442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:33.918873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:33.922309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:33.922831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:33.923302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:33.925529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:33.927950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:33.930051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 206/426] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:33 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:33.939932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:34.068730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e8dcf-1fac-43a8-92f5-1b37bbe4992d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:34.110588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073be40-f52f-4d5c-b910-d2fa1a3857bb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:34.127528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:34.128408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.128692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.128962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] PATH_INFO: `/v3/projects/8770b556c0c24d1b85b244d3070b0330` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.130133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] RBAC: Authorizing `identity:get_project(project_id=8770b556c0c24d1b85b244d3070b0330)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:34.131542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:34.133223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 221/427] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:33 2022] GET /identity/v3/projects/8770b556c0c24d1b85b244d3070b0330 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:34.138509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:34.169536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:34.170396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.170684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.170912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.172973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:34.174416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:34.177716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 207/428] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:33 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:34.183060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:34.325087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e21a3c-7fb8-4d67-b852-552dc9afc7a9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:34.336473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418dbea-52ae-44d5-85a8-970e95509658 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:34.387098 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:34.388382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.390493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.391085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.393356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:34.397315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:34.400612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 222/429] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:34 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:34.402051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:34.402988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.403469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.403942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.405968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=8770b556c0c24d1b85b244d3070b0330) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:34.407157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:34.408647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:34.410139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7ad22-a1e0-4c53-a644-225e2d45c253 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:34.421775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 208/430] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:34 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 611 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:34.512235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8a692-af04-4abe-a65a-bf53dba9965c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:34.564270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:34.564946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.565226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.565494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.566348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=8770b556c0c24d1b85b244d3070b0330) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:34.566656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:34.568114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:34.578528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116024d-6ae2-4e4d-8ba2-3ac4366efa0e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.578810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116024d-6ae2-4e4d-8ba2-3ac4366efa0e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.578994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116024d-6ae2-4e4d-8ba2-3ac4366efa0e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.580636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 209/431] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:34.581814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 223/432] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:34 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=8770b556c0c24d1b85b244d3070b0330&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:34.586205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.586492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.586768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.637471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:34.638697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:34.734674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 210/433] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:34.762149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:34.908889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ed8ce-fcd6-4276-acad-e0e43aa56f3c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:34.943692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e3ad-14f6-4ef5-930f-f44edbff919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:34.944525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e3ad-14f6-4ef5-930f-f44edbff919e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:34.944806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e3ad-14f6-4ef5-930f-f44edbff919e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:34.945081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e3ad-14f6-4ef5-930f-f44edbff919e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:34.956258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e3ad-14f6-4ef5-930f-f44edbff919e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3884e3ad-14f6-4ef5-930f-f44edbff919e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:34.963861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:34.964796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:34.965793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 224/434] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:34.971681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.072055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0bed-ee9e-4fc1-bd63-462693386c3a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.072453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0bed-ee9e-4fc1-bd63-462693386c3a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.072672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0bed-ee9e-4fc1-bd63-462693386c3a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.074280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 225/435] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:35.078907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1f76c-b2de-489d-b191-981a849bc6df None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.083758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.084435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.084917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.103965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:35.105525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:35.125358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.126412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.126707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.126986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.127730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:35.128033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.129615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.142683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 211/436] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:31:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:35.148660 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.188875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 226/437] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:35.213566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.278316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87c5de-fd1a-41d2-9cca-c45822a9fde3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.313608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655e3f1-7fed-4aec-ae73-64f83d6c8f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.314257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655e3f1-7fed-4aec-ae73-64f83d6c8f18 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.314533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655e3f1-7fed-4aec-ae73-64f83d6c8f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.314807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655e3f1-7fed-4aec-ae73-64f83d6c8f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.322948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442a747-7350-4193-aeae-2753a552d0c5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.355292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102f272-8a7d-468f-b7bc-e4243bd2d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.356093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102f272-8a7d-468f-b7bc-e4243bd2d62d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.356405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102f272-8a7d-468f-b7bc-e4243bd2d62d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.356679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102f272-8a7d-468f-b7bc-e4243bd2d62d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.367102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102f272-8a7d-468f-b7bc-e4243bd2d62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5102f272-8a7d-468f-b7bc-e4243bd2d62d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:35.376058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:35.377147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:35.378009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 227/438] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:35.378662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020096d-6e04-4bcd-b43a-2b0d07f31c45 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.378846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020096d-6e04-4bcd-b43a-2b0d07f31c45 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.379129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020096d-6e04-4bcd-b43a-2b0d07f31c45 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.379748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655e3f1-7fed-4aec-ae73-64f83d6c8f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.380517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 228/439] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:35.381224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655e3f1-7fed-4aec-ae73-64f83d6c8f18 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.385297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020096d-6e04-4bcd-b43a-2b0d07f31c45 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.480991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 212/440] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:35.482022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.482309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.483123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.491114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020096d-6e04-4bcd-b43a-2b0d07f31c45 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.500183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:35.501348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:35.522722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.523377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.523652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.523923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.524693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:35.524990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.526345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.535068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 229/441] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:35.535871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.565648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 213/442] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:35.566476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.634723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f081c7c-f24e-42ae-a638-5135a3cb4bc0 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.664304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93340d6-9fd0-46ce-97f1-256d7441cbce None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.667684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.668571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.668852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.669125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.670343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.671751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.673457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 230/443] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:35.674208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.696394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.697029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.697309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.697586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.759447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.760836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.769545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12f02-4557-40d5-9dd6-5c92a15eb948 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.801108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.801944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.802266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.802540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.812777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:35.820236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:35.821125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:35.821905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 231/444] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:35.822557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.854541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 214/445] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:35.855371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.913943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47be03d-7a63-4da3-bde5-9eba305597fb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.944780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.945613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.945847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1744361-58cd-43ad-bfe9-0b8e9b3b4dfb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:35.945953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.946150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.947300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.949062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.950860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 232/446] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:35.951520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:35.977727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:35.978380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:35.978657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:35.978926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:35.979665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:35.979966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:35.981356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:35.989523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 215/447] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:35.990540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.045007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18787d-6e68-48bd-bd30-0885cf0f1cce None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.077512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.078367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.078650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.078922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] PATH_INFO: `/v3/projects/99f7b5bc85b1406083bf0c40dee773d1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.080093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7b5bc85b1406083bf0c40dee773d1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.080716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10cf4-1ca1-4788-99a9-6f7e4212d9e5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.081589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.084199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 233/448] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/projects/99f7b5bc85b1406083bf0c40dee773d1 => generated 302 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:36.084199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.112919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.113558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.113838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.114110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.114925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=039b03887b8b49dfab13afaf1d54bfc0, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:36.115472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.116935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.122408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 216/449] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:35 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=039b03887b8b49dfab13afaf1d54bfc0 => generated 611 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:36.123202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.187751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bbf42-bb6a-4fa9-bd55-85c3f1826045 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.224777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.225452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.225738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.226019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.228647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af794ae-bbf2-40ac-9679-c4c13694b8eb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.267740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.268642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.268923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.269189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] PATH_INFO: `/v3/groups/3e1d433709624bfda8483d85ffe1b0f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.270422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] RBAC: Authorizing `identity:get_group(group_id=3e1d433709624bfda8483d85ffe1b0f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.271883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.273804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 217/450] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:36 2022] GET /identity/v3/groups/3e1d433709624bfda8483d85ffe1b0f5 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:36.280120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.300322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.301699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.383722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c0de-5a23-4396-9cd6-ff77c7ec56fe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.408976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 234/451] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:36.415012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.419253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.419896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.420278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.420540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.421364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] RBAC: Adding query filter params (group.id=3e1d433709624bfda8483d85ffe1b0f5, role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=99f7b5bc85b1406083bf0c40dee773d1) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:36.421665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.423072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.429095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 218/452] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:36 2022] GET /identity/v3/role_assignments?group.id=3e1d433709624bfda8483d85ffe1b0f5&scope.project.id=99f7b5bc85b1406083bf0c40dee773d1&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:36.527573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05975ac-be54-4a47-b5dc-a86f23de69c6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.564386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.565184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.565469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.565748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] PATH_INFO: `/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.566989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] RBAC: Authorizing `identity:get_project(project_id=ccf7092b8d0b4d6a9522dc9ba6d91833)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.568464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.570204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 235/453] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:31:36 2022] GET /identity/v3/projects/ccf7092b8d0b4d6a9522dc9ba6d91833 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:36.575831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.676508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48910400-8703-4599-9a4f-d7bb67fb3730 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.708598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035ee91-dee5-4d71-94ea-c7e921a51aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.709454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ee91-dee5-4d71-94ea-c7e921a51aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.709739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ee91-dee5-4d71-94ea-c7e921a51aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.710014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ee91-dee5-4d71-94ea-c7e921a51aa5 None admin] PATH_INFO: `/v3/groups/c55b414aa0404ad182715fe53bf5fe41` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.711173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035ee91-dee5-4d71-94ea-c7e921a51aa5 None admin] RBAC: Authorizing `identity:get_group(group_id=c55b414aa0404ad182715fe53bf5fe41)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.712646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035ee91-dee5-4d71-94ea-c7e921a51aa5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.714249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 219/454] 10.222.0.73 () {56 vars in 1180 bytes} [Thu May 12 17:31:36 2022] GET /identity/v3/groups/c55b414aa0404ad182715fe53bf5fe41 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:36.720976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:36.820040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c536a05-db93-47d3-913d-0fc1382db06f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:36.851915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:36.852587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:36.852871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:36.853149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:36.853966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] RBAC: Adding query filter params (group.id=c55b414aa0404ad182715fe53bf5fe41, role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ccf7092b8d0b4d6a9522dc9ba6d91833) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:36.854279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:36.855689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5805b5f-0563-4629-a79b-b5d7b45c7537 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:36.861925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 236/455] 10.222.0.73 () {56 vars in 1376 bytes} [Thu May 12 17:31:36 2022] GET /identity/v3/role_assignments?group.id=c55b414aa0404ad182715fe53bf5fe41&scope.project.id=ccf7092b8d0b4d6a9522dc9ba6d91833&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:39.694862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bc81a-e540-4606-bc4b-2ada62d1a18b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:39.695804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bc81a-e540-4606-bc4b-2ada62d1a18b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:39.696297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bc81a-e540-4606-bc4b-2ada62d1a18b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:39.698049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 220/456] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:39.709772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:39.710661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:39.711128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:39.736108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16909b8-5790-4e78-be68-7a187dde2bf2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:39.736747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16909b8-5790-4e78-be68-7a187dde2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:39.737213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16909b8-5790-4e78-be68-7a187dde2bf2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:39.738991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 221/457] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:39.740189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:39.741839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:39.743752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:39.744281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:39.744770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:39.773221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:39.774423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:39.832658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 237/458] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:39.867206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 222/459] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:39.870310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:39.891702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:39.992816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793ca7b-4b76-4da9-873f-5085c1f0a866 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:39.995381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217bb0-e2b7-427d-aa40-c42ee11070e6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.033147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.033948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.034227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.034500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.037699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.038493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.038775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.039051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.053690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.055155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.055704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.057411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fcf46215-3163-4a0c-96bc-6b51b6cb11d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:40.074191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:40.075096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:40.075930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:40.075930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:40.075930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.075930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:40.077094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:40.078926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:40.080989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:40.080989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:40.080989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.080989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:40.081463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 223/460] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:40.081598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 238/461] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:40.081598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.084192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e72c7-9193-45d4-ae7e-16fe409b369e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.084192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e72c7-9193-45d4-ae7e-16fe409b369e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.084192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e72c7-9193-45d4-ae7e-16fe409b369e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.086880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 224/462] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:40.086880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e72c7-9193-45d4-ae7e-16fe409b369e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.186471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06189-9a43-4b25-8aae-5c45da0b13e8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.196228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e72c7-9193-45d4-ae7e-16fe409b369e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.219648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.220341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.220627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.220905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.221626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:40.221938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.223290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da378-64a3-47e7-9420-0f7d477e140e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.231070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.231725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.232007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.232331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.234239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:40.234548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.235924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.246453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 239/463] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:40.247852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.248251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.248675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.252924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 225/464] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:40.253800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.267108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:40.268352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:40.339068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 240/465] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:40.339918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.369362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8554d6-0627-45f3-be31-376a6dbdf0b4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.405654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6263032-606a-4ab5-bf37-64370d2b8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.406323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6263032-606a-4ab5-bf37-64370d2b8fe9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.406602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6263032-606a-4ab5-bf37-64370d2b8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.406882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6263032-606a-4ab5-bf37-64370d2b8fe9 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.407568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6263032-606a-4ab5-bf37-64370d2b8fe9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.409031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6263032-606a-4ab5-bf37-64370d2b8fe9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.432929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 226/466] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:40 2022] POST /identity/v3/users => generated 248 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:40.434873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae45a-93a8-4f05-82c5-95f070d09968 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.435286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae45a-93a8-4f05-82c5-95f070d09968 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.436266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae45a-93a8-4f05-82c5-95f070d09968 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.440396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 227/467] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:40.441564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae45a-93a8-4f05-82c5-95f070d09968 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.447484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6e890-455c-4466-8a80-b40d6fd77a80 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.488523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55b352-63a7-45f0-8752-df078bedf60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.489200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55b352-63a7-45f0-8752-df078bedf60a None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.489481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55b352-63a7-45f0-8752-df078bedf60a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.489795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55b352-63a7-45f0-8752-df078bedf60a None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.490764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55b352-63a7-45f0-8752-df078bedf60a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.492289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55b352-63a7-45f0-8752-df078bedf60a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.518516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 241/468] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:40 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:40.519825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.520142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.520449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.538329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:40.539551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:40.600281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae45a-93a8-4f05-82c5-95f070d09968 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.613541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 242/469] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:40.637965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.639802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.639969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.640139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.640434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.652846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.654440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:40.673838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:40.674753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:40.675599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:40.675599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:40.675599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.675599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:40.675599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 228/470] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:40.681460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.750804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccaf41-83ad-4af7-92ee-1747d266e8fc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.788256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12324d-6809-4218-8a1e-bc82697def3b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.788929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.789312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.789594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.789870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.802028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.803771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:40.824375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:40.827313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:40.829482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:40.829482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:40.829482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:40.829482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:40.829989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:40.829989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:40.830302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 243/471] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:40.830442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:40.830442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:40.830442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:40.830442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:40.830442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:40.840202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.843311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 229/472] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:40.848892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:40.969267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8bc07-06e4-47b7-b73e-ac313f5844f5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:40.984318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bfe31-494a-4584-b87e-8d663a92667a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:41.023301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:41.023301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.023301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.023598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.024499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:41.024803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:41.027082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:41.031741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333d5a3-68c6-4805-b134-519b5ccae5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:41.032758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333d5a3-68c6-4805-b134-519b5ccae5e3 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.033295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333d5a3-68c6-4805-b134-519b5ccae5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.033758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333d5a3-68c6-4805-b134-519b5ccae5e3 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.034657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333d5a3-68c6-4805-b134-519b5ccae5e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:41.036352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333d5a3-68c6-4805-b134-519b5ccae5e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:41.040043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 244/473] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:41.046715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:41.067507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 230/474] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:40 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:41.156586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8be8d-236e-41cc-98cb-1764d06d37d8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:41.171868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a37a38-f63b-404f-b160-53a65bc4d481 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.172228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a37a38-f63b-404f-b160-53a65bc4d481 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.172460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a37a38-f63b-404f-b160-53a65bc4d481 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.174094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 231/475] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:41.179744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.180028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.180348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.192637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:41.193317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.193694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.193969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.194701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:41.196451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:41.221066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 245/476] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:41 2022] POST /identity/v3/users => generated 249 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:41.225492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:41.226728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:41.347210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 232/477] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:41.392552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:41.509252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7899c-f267-4123-ba2a-6c800c5e40cb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:41.543478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:41.544151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.544495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.544776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.546173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:41.547550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:41.554909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 12 17:31:41.554909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 12 17:31:41.554909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [parameters: {'id': 'f37ba4aaa9924fc1b63420aadbd43fd5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 12 17:31:41.554909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() May 12 17:31:41.572720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return ref.to_dict() May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 17:31:41.574708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 12 17:31:41.576665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 12 17:31:41.577817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application c = connection._execute_20( May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 17:31:41.579415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 17:31:41.580774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'id': 'f37ba4aaa9924fc1b63420aadbd43fd5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:41.581715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 12 17:31:41.582810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:41.585142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 246/478] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 12 17:31:41 2022] POST /identity/v3/roles => generated 144 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:41.585221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:41.696854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc5534-af02-4491-812d-6c61f8e4af3c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:41.731381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ae21-6af3-46dd-87d6-428755e6ec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:41.732210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ae21-6af3-46dd-87d6-428755e6ec72 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.733115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ae21-6af3-46dd-87d6-428755e6ec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.733428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ae21-6af3-46dd-87d6-428755e6ec72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.744125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ae21-6af3-46dd-87d6-428755e6ec72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-69c3ae21-6af3-46dd-87d6-428755e6ec72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:41.752939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:41.755036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:41.756122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 233/479] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:31:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:41.759799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:41.928913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72a63-d4b8-4aa6-95c0-cbc02a4b3d93 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:41.973725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:41.974398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:41.974680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:41.974955 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:41.975715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:41.976021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:41.977420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b3d88-412f-4d4f-a5ad-bae56d8bc59c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:41.991862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 247/480] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:43.440964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624f320-2659-45ad-bd9e-e481d40757eb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:43.441490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624f320-2659-45ad-bd9e-e481d40757eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:43.441490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624f320-2659-45ad-bd9e-e481d40757eb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:43.443219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 234/481] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:43.450559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aff28-278b-44b6-9758-65424cdc31b3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:43.451042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aff28-278b-44b6-9758-65424cdc31b3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:43.451178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36aff28-278b-44b6-9758-65424cdc31b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:43.473731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c36aff28-278b-44b6-9758-65424cdc31b3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:43.474923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c36aff28-278b-44b6-9758-65424cdc31b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:43.563093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 248/482] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:43.599051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624f320-2659-45ad-bd9e-e481d40757eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:43.659356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875f2d-f47e-40b0-86f1-16f813e27ed2 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:43.659681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875f2d-f47e-40b0-86f1-16f813e27ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:43.659919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875f2d-f47e-40b0-86f1-16f813e27ed2 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:43.661540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 249/483] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:43.667301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:43.667593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:43.667868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:43.689348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:43.690526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:43.767531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624f320-2659-45ad-bd9e-e481d40757eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:43.828692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 250/484] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:43.830781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:43.831582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:43.831861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:43.832132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:43.851069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:43.853971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:43.860713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:43.861620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:43.862489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:43.868194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 235/485] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:43.872029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:43.993400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479acf4-2f3e-4dd9-b1c8-655ca490ef36 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.034440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94586fa0-253a-4209-bd7c-22856abdd31b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.070841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.071642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.071925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.072231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.086373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:44.091838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.093033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.093316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.093597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.094429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:44.094737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.096132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:44.101452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:44.102340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:44.103135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:44.105513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 251/486] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:44.109507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 236/487] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:44.117183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.118243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.234968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbedb7-3807-4104-a67a-a67b633a3483 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.237782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35b3f8-23b3-49f0-b279-dacbe46283e4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.268144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.269110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.269564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.270034 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.270941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:44.271257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.271615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.272043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.272361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.272639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.273711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822b24f-852e-4028-899d-9678e603d905 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.287332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 237/488] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:44.287498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.287598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ede9f-b98c-4ee8-89d2-9991609979bb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.287598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ede9f-b98c-4ee8-89d2-9991609979bb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.287598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ede9f-b98c-4ee8-89d2-9991609979bb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.287773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.290211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 238/489] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:44.291395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de6775-3bef-4be2-8e03-fcbd74dedcb3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.291682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de6775-3bef-4be2-8e03-fcbd74dedcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.291962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de6775-3bef-4be2-8e03-fcbd74dedcb3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.293705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 239/490] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:44.294167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de6775-3bef-4be2-8e03-fcbd74dedcb3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d1de83db-599b-454d-96ee-1e5ac5fa0de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:44.308502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:44.309403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 252/491] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0748c-e1bb-44a6-aa70-63e39710c131 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0748c-e1bb-44a6-aa70-63e39710c131 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.315094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0748c-e1bb-44a6-aa70-63e39710c131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.330622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-46b0748c-e1bb-44a6-aa70-63e39710c131 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:44.331800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-46b0748c-e1bb-44a6-aa70-63e39710c131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:44.397293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 253/492] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:44.405000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.405000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.405000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.409325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de6775-3bef-4be2-8e03-fcbd74dedcb3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.419187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:44.420923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:44.442221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.443572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.443847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.444113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.454820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.456271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:44.473332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:44.474204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:44.475055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:44.475055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:44.475055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:44.475055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:44.479090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 240/493] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:44.479738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.496192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 254/494] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:44.496192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.575459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ed268-c914-45c7-bdab-5faaf5f305e3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.603825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f7508-c18d-4491-a0a9-3616d4e95f79 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.608527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.609186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.609463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.609730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.610452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:44.610758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.613930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.628327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 241/495] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:44.628327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.647781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.648604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.648879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.649149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.659365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:44.666898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:44.668046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:44.668895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 255/496] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:44.669538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.727920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634cc57-90e9-437b-929f-f2507181cb39 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.760606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.761252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.761528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.761796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.762505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:44.762807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.764191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.769169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0abd0-d878-44c5-8436-2da5ac156b70 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.779074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 242/497] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:44.779074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.809223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.810045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.810330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.810787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.822446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:44.830345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:44.831248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 256/498] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:44.836786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.883460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747abebc-f686-44a9-b4d7-c84cee666680 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.916390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.917148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.917426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.917698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.925991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.927428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.927715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3c111-f35c-4846-a65f-4a710b78f378 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:44.944185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:44.945289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 243/499] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:44.950839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:44.968704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:44.969360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:44.969638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:44.969908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:44.970646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:44.970947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:44.972329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:44.980013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 257/500] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:44.980814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.052439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8cf1-30ce-4d3c-9277-c3f247aa9f19 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.071939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db775-e1e9-4f0d-9a0b-d46adf6962a8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.084605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.085325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.085605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.085874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.093134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.094567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.103106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.103750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.104045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.104331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.105058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:45.105357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.106677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:45.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:45.109203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 244/501] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:45.110112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.116315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 258/502] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:45.117185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.207371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e566c0-c901-48a8-bc8e-1b91055b322e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.210694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2e564-2e0b-45c3-9b14-4b0b76738719 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.238196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.238847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.239178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.239390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.240236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:45.240788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.242191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.242316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.243036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.243318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.243593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.254272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.255694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8d15a4b9-a982-49bc-85de-5c27f4da62a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:45.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:45.272905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:45.273766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:45.273766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:45.273766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:45.273766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:45.273766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 259/503] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:45.274529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f744-74a1-49e1-a7b4-4b14535d0cd9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.274759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f744-74a1-49e1-a7b4-4b14535d0cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.275038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f744-74a1-49e1-a7b4-4b14535d0cd9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.276291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 260/504] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:45.276965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f744-74a1-49e1-a7b4-4b14535d0cd9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.281683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 245/505] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:45.282688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.372449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f744-74a1-49e1-a7b4-4b14535d0cd9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.377761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d421a-71be-47dc-a9d3-ccf96351e7aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.403523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.404218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.404518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.404793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.405627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:45.405932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.407336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226487-deee-4b92-8fa3-680005bb466d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.408995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.409763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.410042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.410309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.419761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.421721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:45.439254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:45.440264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:45.441173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:45.441173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:45.441173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:45.441173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:45.441173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 246/506] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:45.441535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.443839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 261/507] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:45.445464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.445603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.445888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.462094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:45.463277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:45.525542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 262/508] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:45.526356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.537755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ca00-cd5c-4fe4-a5e2-8140dbd98ed6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.569679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.570329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.570604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.570873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.571585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:45.571885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.573277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.589535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 247/509] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:45.589535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.623653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838eb0a-333b-4958-b2b8-b42410816596 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.655889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.656560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.656837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.657105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.678598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557706a-a422-49f3-88ad-e3e17e45b0b0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.710748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.711397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.711679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.711953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.712709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:45.713016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.714390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.718479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.719897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.725296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 248/510] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:45.726101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.811056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 263/511] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:45.811867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:45.818107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd86b35-2393-4e2a-a2b3-175ea96b70af None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.849257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.849899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.850177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.850468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.903706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912f7e9-6318-413d-99f4-3fdab9253ac9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:45.918105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.919552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.936596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:45.937373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:45.937649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:45.937916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:45.947578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:45.949257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:45.967523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:45.968588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:45.969556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:45.969556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:45.969556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:45.969556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:45.969556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 264/512] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:45.969886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.013149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 249/513] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:46.014239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.062884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0192-593c-4aec-9e18-38cce5dd5c83 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.093473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.094255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.094536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.094813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.101876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.103300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.105730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28b30e-8f11-4aee-8a61-fdf5ed28d5ff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:46.118942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:46.120010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 265/514] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:46.121022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.140635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.144351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.144351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.144351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.150690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.152114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:46.166488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:46.168256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 12 17:31:46.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:46.173140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 250/515] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:46.173576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.218294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7bc6e-fe39-42ba-8374-f1e70ee1ac62 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.250349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.251129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.251408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.251680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.261755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.263203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.265336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be7d40-7e89-42cb-ac31-a4e792523f7b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:46.279892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:46.280883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:46.281722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:46.281722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:46.281722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:46.281722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:46.281722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 266/516] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:46.281990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.297820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.298468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.298745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.299013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.299756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:46.300055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.301429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.313218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 251/517] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:46.313905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.375450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a0d9a-80a2-4e01-b420-e22a0b74c141 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.405501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c237f5-e6d3-454e-9785-4fd19800c40d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.406479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.407248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.407527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.407797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.417704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.419074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:46.436339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:46.437259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:46.438087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:46.438087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:46.438087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:46.438087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:46.438294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.438378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 267/518] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:46.438444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.438518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.438585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.438585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.439409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:46.439716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.441146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.458669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 252/519] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:46.459483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.550705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc36d4e-3b60-44d5-bc17-257385d513ee None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.551642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f97c-9736-475a-a1ba-f567f5b16a7d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.584014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.584409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.584522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.584758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.585025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.585141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.585327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.585596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.585858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:46.586158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.586305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:46.586607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.587566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.587952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.598856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 253/520] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:46.599631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.608653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 268/521] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:46.608653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.692767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5208b0-d7f4-4c3b-90fd-ea450f49136a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.697228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739ebbb-9535-4ddf-a12d-3b364ffa8d95 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.722703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.723354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.723630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.723899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.724618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.726018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.728223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.728872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.729153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.729422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.730131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:46.730431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.731751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.743397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 269/522] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:46.744210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.751105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 254/523] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:31:46 2022] POST /identity/v3/users => generated 247 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:31:46.751877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.851536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519e32a-1895-4d65-8422-ebb37ad5d55d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.854730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc9619-7de7-4ca6-810c-b6289227ce17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:46.886737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.887394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.887677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.887954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.889699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:46.890502 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:46.890779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:46.891043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:46.898508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.899989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:46.915512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:46.916714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:46.917568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:46.922221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 255/524] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:46.922970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:46.957331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:46.958708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.022151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878529b-f706-42d3-95e2-2d05b0c54df8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.054869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.055515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.055795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.056065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.058152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 270/525] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:47.058778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.124649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.126167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.155996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff19bc-6965-4e50-a36b-720e92c104f5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.188779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.189562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.189844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.190115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.196688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.198123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:47.212360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:47.213211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:47.214061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:47.216318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 271/526] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:47.217962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.221950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 256/527] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:47.222966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.316414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbb6c2-aba1-436e-963f-4fac16033a08 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.316767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e743ad-a77f-4b21-b674-39bc0d7dba6f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.348286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.348945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.349224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.349495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.349778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.350274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:47.350558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.350651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.350834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.351108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.351982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.361527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.363013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.370063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 272/528] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:47.371038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:47.380455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:47.381305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:47.382174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:47.382174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:47.382174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:47.382174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:47.384489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 257/529] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:47.385493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.470362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f6294-ae66-4100-9740-e8413dbd04a9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.480181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebba14-a91b-4d50-9472-c6862843d2b7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.504047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.504840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.505015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.505269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.506096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:47.506402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.507839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.514577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.515355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.515632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.515899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.524677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 273/530] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:47.524677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.525840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.527282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:47.545511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:47.546549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:47.552905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:47.552905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:47.552905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:47.552905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:47.552905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 258/531] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:47.552905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.624733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130844b-4c20-4342-bed0-79041b26065b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.647564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae04e1-0fc9-4143-a2f2-c334f6042716 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.658731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.659384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.659673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.659950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.660816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=71263b9cdb9d43ca9ce74c3e2069265f) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:47.661125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.662555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.667852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 274/532] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/role_assignments?user.id=71263b9cdb9d43ca9ce74c3e2069265f&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:47.668665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.691797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.692890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.693426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.693894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.694883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:47.695389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.697044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.710617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 259/533] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:47.711914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.780591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a20f61-6279-43b8-8029-b29b9c3a30a8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.816414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.817072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.817357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.817629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.818483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=62c3b98740d14ab49d91783086150e88) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:47.818778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.820256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.822728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692f1f-0084-43b3-88a5-c3dec5b1e9eb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:47.832200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 275/534] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/role_assignments?user.id=62c3b98740d14ab49d91783086150e88&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:47.832200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.900919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:47.901591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:47.901873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:47.902148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:47.902883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:47.903190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:47.905242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:47.923930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 260/535] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:47.929409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:47.973034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b1896-a7b6-40ce-9fd6-1898018d60bc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:48.009975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:48.010778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:48.011058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:48.011330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:48.019577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:48.021103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:48.036900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:48.037775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:48.038617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 276/536] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:48.043924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:48.120077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be4dc7-8efd-462d-9aea-02c21928de47 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:48.145717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b43eb-ee6a-4b50-b9cc-fb9c1635cb7d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:48.161842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:48.162983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:48.163471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:48.163985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:48.177890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:48.179867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:48.180203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:48.180917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:48.181203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:48.181481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:48.182269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:48.182575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:48.184082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:48.196638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 277/537] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:48.202900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:48.206178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:48.207000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:48.207824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:48.216518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 261/538] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:48.218487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:48.302460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ffdaf-99b7-4c51-83d2-9dea3a4da599 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:48.337277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:48.337927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:48.338208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:48.338478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:48.339306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=5621c3ccfb8040748f8adef3dc810c1b) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:48.339616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:48.341063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:48.346975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 278/539] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:48 2022] GET /identity/v3/role_assignments?user.id=5621c3ccfb8040748f8adef3dc810c1b&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:48.353496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd2dc-f8c7-42cb-9d50-6b287b9e1619 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:48.397779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:48.398855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:48.399339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:48.399829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:48.400875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:48.401423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:48.403132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dec9a1-73a8-4362-80f6-56aa2399e1c6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:48.424696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 262/540] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:48.430837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:48.562687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9017d-cdcf-41af-a381-d9ba07fef7f4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:48.605388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:48.606053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:48.606377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:48.606659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:48.607531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=6076a5f71eed4f8fa47d74a79d6fa709) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:48.607843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:48.609435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc748c-e0f4-479b-8b09-4f9c623c81af None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:48.621168 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 279/541] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:48 2022] GET /identity/v3/role_assignments?user.id=6076a5f71eed4f8fa47d74a79d6fa709&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:49.201882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6931f-c4f5-4175-935a-c8134d4ea1f8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:49.202517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6931f-c4f5-4175-935a-c8134d4ea1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:49.202517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6931f-c4f5-4175-935a-c8134d4ea1f8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:49.204062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 263/542] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:49.209708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:49.210147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:49.210336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:49.233776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:49.234986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:49.318886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 280/543] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:49.362276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6931f-c4f5-4175-935a-c8134d4ea1f8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:49.479487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6931f-c4f5-4175-935a-c8134d4ea1f8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:49.519764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:49.520617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:49.520902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:49.521175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:49.537006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:49.544685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:49.545653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:49.546449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 264/544] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:49.551922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:49.694480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84d45-0e54-4de7-a933-c66ee24b509b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:49.731575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:49.732222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:49.733165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:49.733478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:49.734231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:49.734535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:49.735890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:49.744280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 281/545] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:49.750512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:49.899598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702ae2-ba4f-4cdb-86dc-abe08742cb49 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:49.946007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:49.946793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:49.947072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:49.947345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:49.962326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:49.963787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:49.985302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:49.986182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:49.987006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:49.987006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:49.987006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:49.987006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:49.987006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 265/546] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:49.993236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:50.114358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f7221-9ef3-46e2-9f50-e92a74dc52d3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:50.156109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:50.156802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.157080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.157350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.158123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:50.158648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:50.160344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:50.179334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 282/547] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:50.185246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:50.342925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7443b-f5fc-44b1-8e79-3536ed361007 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:50.390394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:50.391185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.391467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.391741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.406188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:50.407644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c94e72b6-948e-4e87-809a-ed568a11aaef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:50.424889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:50.427016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:50.429225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 266/548] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:50.432152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:50.517572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd05796-840c-429c-affe-710391ce46f7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.517572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd05796-840c-429c-affe-710391ce46f7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.519475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd05796-840c-429c-affe-710391ce46f7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.519475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 267/549] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:31:50.528729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eff22-8beb-4cec-a9e9-baa44a38a5ac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.529030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eff22-8beb-4cec-a9e9-baa44a38a5ac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.529848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eff22-8beb-4cec-a9e9-baa44a38a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.568661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-808eff22-8beb-4cec-a9e9-baa44a38a5ac None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:50.570287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-808eff22-8beb-4cec-a9e9-baa44a38a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:50.577099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2618-5902-404d-9201-8cdc77c0398a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:50.613170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:50.614202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.614705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.615185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.616507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:50.617037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:50.619369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:50.637325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 283/550] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 12 17:31:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:50.643544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:50.723481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 268/551] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:50.728940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655374-82a4-41db-90ee-871649b2e75a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.729224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655374-82a4-41db-90ee-871649b2e75a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.729496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e655374-82a4-41db-90ee-871649b2e75a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.731020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 269/552] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:50.737898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.738967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.739248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.769582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:50.770822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:50.794387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe803898-0b41-482d-a43d-e795800dccac None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:50.829897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:50.830645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:50.830993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:50.831336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:50.853830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 270/553] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:50.855013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:50.929041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:50.930958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:50.977188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eda59-1708-496e-ad58-3ac6367e5b43 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.012925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.013912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.014227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.014683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.025418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:51.034291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:51.035195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:51.035973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:51.040333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 271/554] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:51.041203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.055541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 284/555] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:51.056794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.149885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04899d-329f-42cc-92e8-1dc7a3ccfade None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.167762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96484ee3-6049-40d4-8ad5-57d98dc716db None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.186496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.187298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.187646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.187982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.198451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:51.204577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.205244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.205527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.205805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.206555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:51.206864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:51.207370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:51.209178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:51.211089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:51.212560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.218850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 272/556] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:51.218850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.219286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 285/557] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:51.219602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.325425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed5cf1-b198-41e0-b44d-1d21b9c0bfcf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.329078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972f88-71cb-424f-97a5-3a6700839311 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.361035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.361825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.362106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.362379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.363832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.364687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.365292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.365829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.366577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:51.366887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.368327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.373392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.374928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.382969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 286/558] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:51.382969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d74f9e68-66aa-4ac4-8865-55e731d08371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:51.393058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:51.393952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:51.394803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:51.394803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:51.394803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:51.394803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:51.394803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 273/559] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:51.395753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a585993-780e-431d-9e55-82d56e9657c7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.395753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a585993-780e-431d-9e55-82d56e9657c7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.396004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a585993-780e-431d-9e55-82d56e9657c7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.398249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 274/560] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:51.398249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a585993-780e-431d-9e55-82d56e9657c7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.483020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626800a3-3fad-44f8-9724-33fc44e03040 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.495172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a585993-780e-431d-9e55-82d56e9657c7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.514368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.515220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.515502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.515775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.526066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.526419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.527193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.527473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.527580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.527746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.538915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.540373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:51.544406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:51.545870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:51.546755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:51.546755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:51.546755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:51.546755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:51.549440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 287/561] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:51.549440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-99205b99-390e-4f18-bec6-8af3d93f5b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:51.556668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:51.557670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:51.559099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:51.559099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:51.559099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:51.559099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:51.559099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 275/562] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:51.559099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8837-7e6b-406d-b938-0b75dcfcd5df None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.559427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8837-7e6b-406d-b938-0b75dcfcd5df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.559504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8837-7e6b-406d-b938-0b75dcfcd5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.576083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1aad8837-7e6b-406d-b938-0b75dcfcd5df None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:51.577850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1aad8837-7e6b-406d-b938-0b75dcfcd5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:51.641037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 276/563] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:51.642438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f923-685c-412b-a8d7-42c9574a92ad None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.642726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f923-685c-412b-a8d7-42c9574a92ad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.643001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f923-685c-412b-a8d7-42c9574a92ad None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.644264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 277/564] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:51.645083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f923-685c-412b-a8d7-42c9574a92ad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.649108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35353-4d88-4496-b9b7-6fdbb7d5fe71 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.680756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cae87-301b-4162-9f03-08db39be553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.681414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cae87-301b-4162-9f03-08db39be553c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.681691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cae87-301b-4162-9f03-08db39be553c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.681964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cae87-301b-4162-9f03-08db39be553c None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.682680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cae87-301b-4162-9f03-08db39be553c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:51.682987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cae87-301b-4162-9f03-08db39be553c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.684363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cae87-301b-4162-9f03-08db39be553c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.696135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 288/565] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:51.696942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cae87-301b-4162-9f03-08db39be553c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.748604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f923-685c-412b-a8d7-42c9574a92ad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.780078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.780821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.781162 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.781493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.782265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:51.782631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.784052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab391a-83f4-4d7f-8fbd-39a3ec55b52c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.789301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cae87-301b-4162-9f03-08db39be553c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.800219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 278/566] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:51.800219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.800219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.800219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.813322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:51.814595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:51.822876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.823532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.823816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.824085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.824833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:51.825138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:51.826590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:51.837462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 289/567] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:51.838257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.877614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 279/568] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:51.878432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:51.945691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd79f32-12e6-4a84-b9a7-579a4eca3066 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.975566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272895-615f-4fcf-8dec-9d460200179a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:51.978601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:51.979389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:51.979672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:51.979946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:51.990051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:51.997877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:51.998984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:51.999811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 290/569] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.000721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.011972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.012797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.013081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.013356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.020974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.022449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:52.036902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.037901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:52.038917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 280/570] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.039383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.099841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999d514-58fe-4027-b801-2c97d8f6c442 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.130426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.131201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.131477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.131746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.137007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c343f-4e11-43be-90d4-0ad191b94882 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.139292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.140753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:52.154714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.155615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 291/571] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.156970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.174864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.175647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.175929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.176228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.183516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.184975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:52.199377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.200300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:31:52.201419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:52.205498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 281/572] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.205498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.257214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df335ee0-1c67-41b1-9707-eaa19f4c0f95 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.287904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.288700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.288978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.289245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.298376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7835-51b7-4984-b5c3-5c95ce9a7e1c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.299212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:52.306657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.307568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:52.308487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 292/573] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.309181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.330142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.330798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.331079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.331352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.332088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:52.332423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.333780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.346436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 282/574] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:52.346436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.410101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1b71f-6dbd-43a5-a082-5364d0524f69 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.437654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3792e-4a49-4175-9587-b49b05bc147e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.441552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.442196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.442475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.442744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.443526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:52.443829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.445261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.457013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 293/575] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:52.457859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.470356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.471021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.471302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.471578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.472444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:52.472755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.474164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.490770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 283/576] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:52.491554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.549291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928931-3300-4817-8094-ac01d2e8c1c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.580946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.581607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.581907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.582172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.583002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:52.583308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.583905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d795-69fe-4a00-aa04-6682c9b3a8dd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.584751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.601637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 294/577] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:52.602432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.619398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.620127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.620429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.620703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.621445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:52.621749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.623620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.631526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 284/578] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:52.632327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.696025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafac7f-b6dd-48cc-874f-2a57c2a520a1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.723181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6604579-ace0-4a0e-ba0f-117535a07606 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.727877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.728686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.728967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.729240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.739442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.741161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.755235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.755873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.756154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.756453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.757271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=972f23cd94bf4051825701d2aaddfe6d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:52.757580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:52.758369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.759391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.760612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:52.760612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:52.760612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:52.760612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:52.760897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.764187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 295/579] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.764858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.776327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 285/580] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/role_assignments?user.id=972f23cd94bf4051825701d2aaddfe6d&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:52.776327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.876679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124425f-4732-4d7a-89e0-9ff04b844470 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.882579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74682a-1881-4000-abc6-3227d41b83b1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:52.911412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.912227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.912515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.912785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.918522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:52.919379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:52.920271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:52.920544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:52.924775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.926246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.931572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:52.933078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:52.944350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.945438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.946517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:52.946517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:52.946517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:52.946517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:52.950850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 286/581] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.951641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:52.952066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:52.953038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:52.953933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:52.953933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:52.953933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:52.953933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:52.953933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 296/582] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:52.954610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.050615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038427f-d16b-4b21-83ad-e9de0361c7e4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.052852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4e398-7547-41b2-8926-48daa95d88d0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.083663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.084352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.084635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.084903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.085192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.085617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.085917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.086012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.086237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.086505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.087252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.097076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.098486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.098877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 297/583] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.099674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:53.120828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:53.121771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:53.122732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:53.122732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:53.122732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:53.122732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:53.127277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 287/584] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:53.128084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.204735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6464cf-31ac-4f9d-8ca6-be3274f6ce70 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.222613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d847e3-98c8-4f7b-b4a3-a918c8d1db72 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.238533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046575a-51db-4062-9784-2de3a760842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.239794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046575a-51db-4062-9784-2de3a760842b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.239794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046575a-51db-4062-9784-2de3a760842b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.239794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046575a-51db-4062-9784-2de3a760842b None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.240759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046575a-51db-4062-9784-2de3a760842b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.241156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046575a-51db-4062-9784-2de3a760842b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.242428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046575a-51db-4062-9784-2de3a760842b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.261050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 298/585] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.261968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046575a-51db-4062-9784-2de3a760842b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.265257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.266259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.266259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.266454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.267192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.267476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.268885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.281543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 288/586] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.282367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.361559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046575a-51db-4062-9784-2de3a760842b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.376764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ff660-2b25-405a-9841-2575fa7f6905 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.393778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.394532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.394814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.395087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.403173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.405270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.409583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.410240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.410522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.410794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.411507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.411810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.413179 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:53.419077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:53.420855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 12 17:31:53.422013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:53.424491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 299/587] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:53.425559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.430325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 289/588] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.431327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.530354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ef59-8001-4247-9f4e-e9d2f96bcfa1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.531601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59340a1-afb6-4ab1-95e8-c3a5d66434d3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.564219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.564481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.565201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.565312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.565482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.565756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.565856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.566198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.572763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.574205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.586190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.587795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:53.588986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:53.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:53.590963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 300/589] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:53.591681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:53.609577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:53.610519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:53.611430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:53.615735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 290/590] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:53.616637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.693257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d154bfc-b084-4cb7-b510-75251628b778 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.718015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21174-7587-474a-a22d-d1b90a6cdf75 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.726167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.726831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.727115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.727392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.728275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.728589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.730022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.747117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 301/591] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.749246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.756030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.756949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.757226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.757491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.765597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.767050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:53.781759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:53.782688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 291/592] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:53.783722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.855050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84ab7d-92b9-4c34-9f50-ce142e406222 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.894523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.895474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a054ea-de3b-4604-9fb9-7f0d774be872 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:53.895733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.896229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.896704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.898633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.899144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.900784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.915832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 302/593] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.916532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:53.934171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:53.934859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:53.935142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:53.935465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:53.936435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:53.936749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:53.938166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:53.951854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 292/594] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:53.952548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.021478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008b455-a63d-4a80-bc40-2a48b1149c08 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.050657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dc6cf-d196-467e-acee-d0b0dedf268b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.055641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.056467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.056752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.057029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.067929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.069415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.083873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.084574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.084857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.085132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.085956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:54.086263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:54.086718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:54.087587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:54.089169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:54.089169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:54.089169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:54.089169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:54.089412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.089497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 303/595] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:54.093072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.113222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 293/596] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 12 17:31:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:54.113222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.196420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9730c-883f-4bd4-8552-db948bbcf85d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.209562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfb655-51c7-4271-b0f1-cdd9a3ffa8ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.231999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.233431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.233713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.233985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/62c3b98740d14ab49d91783086150e88/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.243003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.243878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.244187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.244447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/71263b9cdb9d43ca9ce74c3e2069265f/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.248683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=62c3b98740d14ab49d91783086150e88, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.250290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.259548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=71263b9cdb9d43ca9ce74c3e2069265f, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.261081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.268305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 304/597] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:54 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/62c3b98740d14ab49d91783086150e88/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:54.268305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.282434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 294/598] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:54 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/71263b9cdb9d43ca9ce74c3e2069265f/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:54.283555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.411232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe8afc-1cf0-4969-9c50-f75c64246352 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.448568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.449500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.449845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.450185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.462587 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.464059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:54.484691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:54.485872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:54.486980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:54.486980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:54.486980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:54.486980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:54.492063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 295/599] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:54.493300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.496416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c250604-8fd1-427d-8112-0f424528105c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.534390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.535067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.535351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.535626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.536394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:54.536703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.538115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.550720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 305/600] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:54.556129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.599400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f086e84-79b7-4102-92f7-50fbc4a56c1a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.634602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.635356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.635696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.636036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.636854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:54.637222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.639267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.655991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 296/601] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:54.657296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.662743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b027f1-b97f-4c01-a03b-4bdb2e05911b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.697826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.698618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.698905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.699180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.707133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.708625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:54.723228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:54.725205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:54.727125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 306/602] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:54.732343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.768009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347665a-a06b-4b42-aa5f-7918f269b803 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.816245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.818089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.819118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.820441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.834754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.837647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:54.862361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:54.863495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:54.864636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:54.870832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 297/603] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:54.872277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:54.892732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b3b1-6444-4178-9071-02480d5c1c23 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:54.930245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:54.931023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:54.931372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:54.931710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:54.932615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:54.932987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:54.934521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:54.947370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 307/604] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:54.954276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:55.028003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16685e-d598-4793-bae7-5f7ddfe383be None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:55.065344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:55.066558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.066911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.067249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.068107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:55.068496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:55.069974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:55.088098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 298/605] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:55.089469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:55.196356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c0000-b02a-455b-b5ef-a6a4906535b5 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:55.198144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ab35-0ead-4b69-97ed-c10955d05c52 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:55.229132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff433f-9e44-476d-9b14-7ecf2dc7ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:55.230069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff433f-9e44-476d-9b14-7ecf2dc7ceea None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.230349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff433f-9e44-476d-9b14-7ecf2dc7ceea None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.230615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff433f-9e44-476d-9b14-7ecf2dc7ceea None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/5621c3ccfb8040748f8adef3dc810c1b/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.240740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:55.241989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.242335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.242820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/6076a5f71eed4f8fa47d74a79d6fa709/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.246204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff433f-9e44-476d-9b14-7ecf2dc7ceea None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=5621c3ccfb8040748f8adef3dc810c1b, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:55.247654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff433f-9e44-476d-9b14-7ecf2dc7ceea None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:55.260429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=6076a5f71eed4f8fa47d74a79d6fa709, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:55.262100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:55.264008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 308/606] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:54 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/5621c3ccfb8040748f8adef3dc810c1b/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:55.265787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2587b-5d80-4a2d-8dcb-8bc30268dffd None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.266228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2587b-5d80-4a2d-8dcb-8bc30268dffd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.266564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2587b-5d80-4a2d-8dcb-8bc30268dffd None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.268447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 309/607] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:55.276332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.276332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.276332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.299930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 299/608] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:55 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/6076a5f71eed4f8fa47d74a79d6fa709/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:31:55.306616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:55.308268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:55.390768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 310/609] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:31:55.414490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:55.532090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4086e-a2e8-464c-9ca5-44bf689ad3ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:55.569773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:55.570849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.571298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.571759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.590260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:55.599701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:55.601256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:55.602286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 300/610] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:55.609551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:55.734805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57598b-7304-440e-83cd-429cb2652128 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:55.770827 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:55.772041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.772491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.772911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.773885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:55.774337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:55.775950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:55.785726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 311/611] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:55.791919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:55.900692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2126c3-808c-4e55-9c4b-31c2cc86d517 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:55.937153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:55.938201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:55.938634 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:55.939076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:55.950550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:55.953187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:55.971764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:55.972950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:55.974071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:55.974071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:55.974071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:55.974071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:55.980260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 301/612] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:55.981971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:56.086450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4346fb-b55e-489a-a666-8f65511c8f93 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:56.120441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:56.121391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:56.121851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:56.122323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:56.123917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:56.124542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:56.126042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:56.139410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 312/613] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:56.144640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:56.250524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d5e11-22cd-4b32-a315-f786339089b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:56.285684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:56.286742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:56.287170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:56.287595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:56.296382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:56.298036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:56.312743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:56.313879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 17:31:56.315061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:56.317329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 302/614] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:31:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:56.329270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:56.442602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003cdf4-9cd7-47b8-a417-44193624601a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:56.482979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:56.483936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:56.484398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:56.484838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:56.485834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:56.486290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:56.487857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:56.510446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 313/615] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 12 17:31:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:56.516298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:56.634249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe249e45-7e4c-4ca3-bd11-7508895b744e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:56.675687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:56.676523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:56.676804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:56.677076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:56.694058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:56.695513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:56.715064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:56.716218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:56.717768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:56.717768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:56.717768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:56.717768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:56.719343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 303/616] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:56.725167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:56.834803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa66163-4043-4ef7-8d85-e231a8adba21 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:56.878075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:56.878980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:56.879439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:56.879862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:56.880755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:56.881205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:56.882693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:56.905961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 314/617] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:56.911678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.031201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969110c-d321-49ed-ad22-c73c6da82cde None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.076108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.077046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.077323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.077590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.085644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:57.087047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c27eb8f4-0bad-49a7-9553-ff6a92400541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:31:57.113543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:57.114511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:31:57.115699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:57.118821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 304/618] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:57.129261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.233183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033831a5-1a9e-4791-912e-a08e5ea833f2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.233862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033831a5-1a9e-4791-912e-a08e5ea833f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.234347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033831a5-1a9e-4791-912e-a08e5ea833f2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.235996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 305/619] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:57.241657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bf9d4-7d94-4237-a0b6-d27e917b0007 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.241948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bf9d4-7d94-4237-a0b6-d27e917b0007 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.242194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bf9d4-7d94-4237-a0b6-d27e917b0007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.272892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1547c0-ad08-4833-92bb-7b647d3fa4a9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.273229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cf6bf9d4-7d94-4237-a0b6-d27e917b0007 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:57.274408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf6bf9d4-7d94-4237-a0b6-d27e917b0007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:57.314125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.314800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.315248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.315528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.316964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:57.317281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:57.319176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379da7a-98bf-4c6d-a1cb-0be2134b82aa None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:57.334486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 315/620] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:57.335871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c48a-c868-4d19-99fa-0d40cc0b15f0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.336103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c48a-c868-4d19-99fa-0d40cc0b15f0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.336405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c48a-c868-4d19-99fa-0d40cc0b15f0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.342153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 316/621] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:57.342931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c48a-c868-4d19-99fa-0d40cc0b15f0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.366341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 306/622] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:57.368110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.368427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.368699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.385930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:57.387134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:57.470206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 307/623] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:57.471099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.499192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c48a-c868-4d19-99fa-0d40cc0b15f0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.533276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.536312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.536312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.536312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/972f23cd94bf4051825701d2aaddfe6d/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.550702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=972f23cd94bf4051825701d2aaddfe6d, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:57.552398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:57.573423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 317/624] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:31:57 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/972f23cd94bf4051825701d2aaddfe6d/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:31:57.575073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.600999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c32d78-6047-4f76-822e-5ca0d72e3ed1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.640498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.642221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.642748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.643247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.656217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:57.664653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:57.665701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:57.666533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:57.669463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 308/625] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:57.676147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.706353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51c0a-f6e6-42a5-87df-e23331243fc2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.752556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.753360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.753634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.753909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.767974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:57.777924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:57.778899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:57.780359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 318/626] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:57.786316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcea60-4d7b-4ff7-af19-c06ac804a9b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.787559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.822789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.823874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.824460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.824963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.825989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:57.826514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:57.828223 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:57.838167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 309/627] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:31:57.844695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.918157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4be9-5701-47b7-b95d-c821a31c79ba None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:57.966445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:57.968045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:57.968598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:57.969083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:57.970038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:57.970540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:57.972136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:57.981353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 319/628] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:57.987187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:57.996531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30daa3-1133-41f8-9a14-bf67d60e292a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.031016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.031815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.032114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.032419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.044127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.045662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:58.064296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:58.065234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:58.066100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:58.066100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:58.066100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:58.066100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:58.072471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 310/629] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:58.073964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.098464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545fc65-c67f-4d14-a0ce-6f01385baaab None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.134668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.135453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.135738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.136017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.147266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.148742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d658f314-5856-45ff-b60b-aae5d41b75c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:58.166324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:58.167264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:58.168392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:58.168392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:58.168392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:58.168392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:58.168392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 320/630] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:58.173486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155e96-4785-459d-a0b0-9ef9f93e1e67 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.173781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155e96-4785-459d-a0b0-9ef9f93e1e67 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.174066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155e96-4785-459d-a0b0-9ef9f93e1e67 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.175670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 321/631] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:58.176446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4155e96-4785-459d-a0b0-9ef9f93e1e67 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.181523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22b6a-d80f-4fa1-a64a-4a09a9c1f43f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.220371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.221460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.221948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.222425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.223365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:58.223894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.225511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508a306-4bf6-4fe0-b8b8-5f1574c1f670 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.238847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 311/632] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:58.245040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.245574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.246080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.263893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:58.265466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:58.292398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4155e96-4785-459d-a0b0-9ef9f93e1e67 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.327249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fdef2-9545-4090-975f-09a334405128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.327967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fdef2-9545-4090-975f-09a334405128 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.328296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fdef2-9545-4090-975f-09a334405128 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.328578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fdef2-9545-4090-975f-09a334405128 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.329392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fdef2-9545-4090-975f-09a334405128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:58.329595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fdef2-9545-4090-975f-09a334405128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.330952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fdef2-9545-4090-975f-09a334405128 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.334982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 312/633] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:58.336146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.354642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 322/634] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:58.360859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fdef2-9545-4090-975f-09a334405128 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.441988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a223f-8d28-4169-9e3c-6ded8e2ef7f8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.473018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fdef2-9545-4090-975f-09a334405128 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.474454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.475753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.475753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.475753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.483353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.484804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:58.499400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:58.500402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 313/635] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:58.501367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.515032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.515901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.516219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.516500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.523980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.525412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b3fa865e-4efc-481a-be77-45d6e5f84f8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:58.540228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:58.541197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 323/636] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5df166-eede-43ef-b739-831afbeba871 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.548227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5df166-eede-43ef-b739-831afbeba871 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.549128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5df166-eede-43ef-b739-831afbeba871 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.549817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 324/637] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:31:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:31:58.550719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5df166-eede-43ef-b739-831afbeba871 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.617593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63fd09-2653-4f1c-b209-bda41c2f1a07 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.650948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672015-603b-400f-9610-68a472f4c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.652451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672015-603b-400f-9610-68a472f4c921 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.652734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672015-603b-400f-9610-68a472f4c921 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.653005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55672015-603b-400f-9610-68a472f4c921 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.655999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5df166-eede-43ef-b739-831afbeba871 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.663824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55672015-603b-400f-9610-68a472f4c921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-55672015-603b-400f-9610-68a472f4c921 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:58.672113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:58.673237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 314/638] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:58.674081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672015-603b-400f-9610-68a472f4c921 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.692402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.693278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.693699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.694105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.695073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:58.695542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.697131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac995f-0d33-4528-ad44-01bc2115fc1f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.715112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 325/639] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:58.717164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.717581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.718029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.735149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:31:58.736364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:31:58.781947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55672015-603b-400f-9610-68a472f4c921 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.800966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 326/640] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:31:58.801780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.827524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.828222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.828503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.828975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.829619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:58.829892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.831312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.848769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 315/641] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:58.849571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.907691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad02887-2e75-445c-b7e2-d45d38bf793c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.943031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.943625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc359d4-16c6-4be3-b0ad-4749ecfb5247 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:58.943709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.943946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.944181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:58.944938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:58.945244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:58.946596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:58.955081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 327/642] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:58.956303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:58.978319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:58.978964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:58.979242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:58.979511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.047773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.049700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.053680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727f2d-e3d8-41de-955f-c8dd1ff2af34 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.085787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.086575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.086858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.087128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.097970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:31:59.105495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:59.106607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:59.107537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 328/643] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:59.108288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.155355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 316/644] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:59.156586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.214789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b450-1c83-4696-a8e2-2a011374c68c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.247247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.248481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.248903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.249188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.253011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef67d38-0efc-4c10-8d57-3108e03c927b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.289754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.290847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.291132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.291685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.303278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.304724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.314644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.316070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:59.322209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:59.323097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:59.324058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:59.324058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:59.324058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.324058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:59.324058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 317/645] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:59.324777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.417110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 329/646] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:31:59.417827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba9b44-a910-4663-832d-6477ff1276d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.417939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.451572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.452256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.452540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.452812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.453557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:59.453856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.455213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.463269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 318/647] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:59.464087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.520285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba4cfa-1950-4642-bd40-1bc686944465 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.553457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.554238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.554771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.554771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.560113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d58df-cb55-425e-bd54-de5a87d30fc3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.566152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.567612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:59.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:59.585386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:59.586273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:59.586273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:59.586273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.586273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:59.586273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 330/648] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:59.586556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.601636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.602298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.602585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.602854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.603574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:59.603876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.605285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.622888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 319/649] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:59.622888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.693710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbd8e-78c6-4a83-91e9-532e9daef929 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.718848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fc68a-a6e7-4c8f-a635-bb8c34d58ba9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.726007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.726796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.727080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.727355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.737843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.739281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.752757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.753867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.754363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.754826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:59.757622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:59.758482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:59.759400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:59.759400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:59.759400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.759400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:31:59.759400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 331/650] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:59.759673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.768037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.769529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:31:59.787291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:59.788205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:59.789111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:31:59.789111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:31:59.789111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:31:59.789111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:59.793003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 320/651] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:59.793701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.860229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000b818-3f8f-4bd3-bfde-48ccb3fe007f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.888581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad854687-efaa-4135-ae21-24ef26c99d40 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:31:59.893283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.894151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.894630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.895077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.895962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:31:59.896463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.897971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.910593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 332/652] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:31:59.911733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:31:59.922892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:31:59.923689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:31:59.923969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:31:59.924272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:31:59.931221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:31:59.932680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:31:59.947077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:31:59.948101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 321/653] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:31:59.954282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.011820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5371d-7da0-4594-9eb9-3a7bc0e0a387 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.049400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.049974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.050253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.050524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.051239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.051540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.052926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.053041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7440517-c63a-4796-8d0e-ad986058769a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.068287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 333/654] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.068287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.089756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.090421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.090736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.091015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.092278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.092584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.093940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.109716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 322/655] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.109716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.161199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a11e9-87db-4298-8686-0569f646263c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.194121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.194902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.195180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.195454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.199740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79e98-6f41-424c-b803-ca314fa0c27e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.202403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.203826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:00.217586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:00.218618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:00.219681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 334/656] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:00.220264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.233238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.233899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.234177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.234448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.235274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.235579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.237037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.254307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 323/657] 10.222.0.73 () {56 vars in 1137 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.255091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.313115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43bfce-fa22-4f20-b74c-bf2177c00d23 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.345157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.345726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfbe57-e2b2-4e6d-9c9f-883e745acbcf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.345936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.346217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.346489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.352929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.354353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-414decc8-81b6-4939-9bd4-61fcad6a34d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:00.368376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:00.369489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:00.370512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:00.372304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 335/658] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:00.373679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343dbf-2e0c-4a76-a928-70e9b7dca556 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.374116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343dbf-2e0c-4a76-a928-70e9b7dca556 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.374617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343dbf-2e0c-4a76-a928-70e9b7dca556 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.375860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 336/659] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:00.376563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343dbf-2e0c-4a76-a928-70e9b7dca556 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.377847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.378639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.379019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.379187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.386314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.387722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:00.402264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:00.403308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 324/660] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:00.405081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.472145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343dbf-2e0c-4a76-a928-70e9b7dca556 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.496739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f2d7f-5ffe-4d6d-b3f7-72484143d938 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.503289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.503934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.504236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.504512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.505288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.505595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.506986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.518532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 337/661] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.519111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.528476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.529719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.530035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.530266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.592720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.594125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.621425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9476fc-0436-44c5-8bee-f41d72008c7c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.653484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.654380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.654654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.654930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.655705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.656007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.657434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d010239-f16d-4cfd-873d-a04639ae42b4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.668608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 338/662] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.669822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.670106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.670376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.685814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:00.686962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:00.687822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 325/663] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:32:00.688679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.749340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 339/664] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:00.750166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.782856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f373f-7b5f-4d6f-ae45-2ec581574f4a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.814845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.815507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.815786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.816053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.816914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.817219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.818627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.834743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 326/665] 10.222.0.73 () {56 vars in 1139 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.835527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.843420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871c86a-97ca-4c98-90f5-32c018bb0296 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.875802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.876473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.876748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.877018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.877832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=71263b9cdb9d43ca9ce74c3e2069265f) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.878136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.879545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.885067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 340/666] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/role_assignments?user.id=71263b9cdb9d43ca9ce74c3e2069265f&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.885819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.938921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef7c87-cb77-414a-980b-67f5bb843586 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:00.974459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:00.975123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:00.975398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:00.975663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:00.976531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=62c3b98740d14ab49d91783086150e88) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:00.976834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:00.978291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:00.984690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 327/667] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/role_assignments?user.id=62c3b98740d14ab49d91783086150e88&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:00.985557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:00.989831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76e1c7-be98-446e-a71c-0cb21dadf053 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.028917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.031126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.031629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.032110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.044396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.046446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:01.065585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:01.066568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:01.067458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:01.067458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:01.067458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:01.067458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:01.069559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 341/668] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:01.070699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.140596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494f7a3-bc84-49a7-803d-d2076151f919 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.178300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.179110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.179389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.179664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.182805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09f53a-5c14-4af5-b8cd-2fc4a51f608a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.191383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:01.199208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:01.200406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:01.201791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:01.204192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 328/669] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:32:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:01.204192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.218343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.219381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.219381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.219718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.298557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.300430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.315557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb5780-6a1c-4213-ab85-dc6bbcfba261 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.350728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.351393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.351670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.351940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.352947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:01.353251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.354622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.370511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 329/670] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:01.371576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.419113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 342/671] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:32:01.421001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.471600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37bbf6-ab6d-454a-965e-2ad01b57f5d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.505867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.506532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.506812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.507080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.507827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:01.508127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.509506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.518089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efede4-00c5-4972-aee6-bde6128377f1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.518381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 330/672] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:01.519166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.553193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.553979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.554261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.554533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.562077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.563552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:01.577328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:01.579373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:01.581548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:01.588190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 343/673] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:01.588190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.632153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ae3cd-de31-4aed-bfb1-4ab3283e77ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.680277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.681016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.681309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.681715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.695789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.697135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6027012-6aba-4c45-a30b-d2e3b7c3f56d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.697299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:01.720287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:01.721248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:01.722566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:01.722566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:01.722566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:01.722566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:01.722566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 331/674] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:01.722566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.735950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.737106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.737589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.738060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.753022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.754741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:01.776844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:01.777781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:01.779367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:01.779367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:01.779367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:01.779367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:01.779367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 344/675] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:01.779367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.835150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2778e6-9ce0-4635-9c64-2fdbc401b391 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.869311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.869651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.869884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.870152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.870992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:01.871295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.872955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.882795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086056f-dd1e-4018-a183-906e1cd25ece None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:01.890520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 332/676] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:01.891143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.918129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:01.919160 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:01.920316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:01.920902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:01.921821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:01.922323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:01.923889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:01.937482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 345/677] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:01.938742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:01.990277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382f37-3422-4ce3-b07d-709fea96d599 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.024350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.025018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.025297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.025584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.026361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:02.026621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.028024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.039167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8951f9-d30f-437f-881b-896e61a92f68 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.040480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 333/678] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:02.041313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:02.074183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.075178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.075697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.076719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.077735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=5621c3ccfb8040748f8adef3dc810c1b) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:02.078245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.079844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.086261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 346/679] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:01 2022] GET /identity/v3/role_assignments?user.id=5621c3ccfb8040748f8adef3dc810c1b&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:02.087277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:02.192142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7572c71-a197-4d1c-ad4a-9ef4e92281ff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.198391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422b6fe-c3a7-4907-835b-686cd01916a2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.229159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.230358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.230888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.231392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.244501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.246358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:02.262377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:02.264136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:02.265227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:02.267715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 347/680] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:02.273490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:02.294497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.296274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.296805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.297274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.312156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.314303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:02.331468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:02.332483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:02.333376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:02.340273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 334/681] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:02.343459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:02.434041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa0457-b1f5-471a-a104-f6fbc319360d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.472210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.472822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.473106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.473379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.474225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:02.474530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.475235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dbb3f-2653-49ca-8bdb-736ffb6456f2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.475963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.499171 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 348/682] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 12 17:32:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:02.504591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:02.515056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.515723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.516007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.516340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.517193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:02.517507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.518926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.533557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 335/683] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:02.539485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:02.657611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716ee3c-4229-42f0-bee4-078e90687869 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.694061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.694720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.695005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.695283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.696114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=6076a5f71eed4f8fa47d74a79d6fa709) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:02.696458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.698053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.712211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 336/684] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:02 2022] GET /identity/v3/role_assignments?user.id=6076a5f71eed4f8fa47d74a79d6fa709&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:02.777414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328606-20f3-4b03-a5c0-3a96cd83f3af None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:02.815640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:02.816835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:02.817484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:02.817957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:02.894132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:02.895939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:02.997353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 349/685] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:32:03.004202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:03.113340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae8ef1-091a-49f0-9297-e13a028ed06e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:03.153512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:03.155056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.155423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.155699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.168052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:03.170129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:03.190249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:03.191193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:03.192115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:03.192115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:03.192115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:03.192115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:03.192115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 337/686] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:03.198380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:03.326455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77eeb09-b0cd-4e7b-8bfc-7c6778317ddc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:03.363167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:03.363894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.364114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.364408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.365132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:03.365431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:03.367133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eb78d-4721-4d89-88fe-d50345646564 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:03.386926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 350/687] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:03.392873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:03.421193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d0225-c49d-4d73-bdf0-56d28c4ec35b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.421493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d0225-c49d-4d73-bdf0-56d28c4ec35b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.421730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d0225-c49d-4d73-bdf0-56d28c4ec35b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.423402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 351/688] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:03.428600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.429122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.429607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.462727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:03.463934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:03.559616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 352/689] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:03.582571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:03.618221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047a6fc-320d-40f0-9c1e-8e3bad233b3c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:03.668774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:03.669954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.670228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.670492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.684216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:03.684216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:03.698461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efea8-3efc-4016-9d8a-4b0edbcba02a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8a0fb0ac-7131-4144-95f7-e8e9730c67a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:03.709497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:03.710618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:03.711715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:03.713485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 338/690] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:03.714849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e7d7-3501-41ee-b785-fde476f34b77 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.715230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e7d7-3501-41ee-b785-fde476f34b77 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.715645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e7d7-3501-41ee-b785-fde476f34b77 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.722629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 339/691] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:03.724336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.724732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.725236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.746005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:03.747299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:03.751921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:03.752940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.753357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.753766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.754553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:03.756492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:03.774297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:03.775432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:03.776476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:03.777650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 353/692] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:03.780300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:03.844189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 340/693] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:03.844189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:03.914031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c217-edd1-4829-95c3-e2964d9f008b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:03.947777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:03.948727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:03.949183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:03.949636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:03.950608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:03.951650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:03.953452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:03.957989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171718e-9f34-44bd-a933-4ca0e3861a37 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:03.974497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 354/694] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:03.974497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.000304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.001169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.001590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.001959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.002772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:04.003174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.004785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.014309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 341/695] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:04.014309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.087625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e58406-08ac-46cb-919a-a451dc9a4208 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.124142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513dcb-df7b-4040-a0f8-ff4b40fc0fc6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.128033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.129035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.129460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.129872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.143264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:04.153002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:04.154727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:04.155757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:04.158700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 355/696] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:04.159655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.167670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.168435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.168865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.169239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.170233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=972f23cd94bf4051825701d2aaddfe6d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:04.170641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.172174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.184191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 342/697] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/role_assignments?user.id=972f23cd94bf4051825701d2aaddfe6d&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:04.184191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.304999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee032c-9779-43bc-b53c-e3751a047441 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.334651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec7356-d538-4e5e-ae92-08dbf624495c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.349733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.350715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.351129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.351523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.354339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:04.354845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.356408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.371492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 343/698] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:04.377447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.423675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.424800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.425297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.425796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.428466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:04.430129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.432010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.439855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 356/699] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:04.524575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f106f3-6c4c-472a-bf2f-e3d78bdd3de2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.567679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.568892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.569412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.569891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.583816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.585743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:04.613459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:04.614425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:04.615336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:04.615336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:04.615336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:04.615336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:04.618501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 344/700] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:04.623709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.739592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76593dc-a08d-49ac-afc5-1a788661b2e4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.780450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.781144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.781435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.781705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.782429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:04.782792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.784151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.796768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 357/701] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:04.802166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:04.910036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7591b-c13f-4fc6-9180-7183414d7921 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:04.944286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:04.945139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:04.945422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:04.945696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:04.953862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:04.955349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f1370857-cb78-46fb-8a09-286179aa6fc2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:04.969993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:04.970976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:04.971904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 345/702] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:04.977144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.073266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e425-82d2-47f1-9e5e-a57e00742fde None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.073894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e425-82d2-47f1-9e5e-a57e00742fde None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.073894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e425-82d2-47f1-9e5e-a57e00742fde None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.075049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 346/703] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:05.083428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09426882-98ce-4ea8-ba51-b9f964c74d6d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.088026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.088333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.088608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.106189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:05.107745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:05.154402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:05.155080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.155357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.155626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.156583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:05.156888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:05.158305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:05.184905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 347/704] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:32:05.199253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 358/705] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:32:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:05.205011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.210856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.327897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b561f75-0779-40a8-9c67-9cff0b3866fc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.367019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b104d7f-adbf-40ee-bd52-03e91c1e20bd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.374896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4068-aad5-4790-b768-765ee93ac9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:05.375885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4068-aad5-4790-b768-765ee93ac9cc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.376400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4068-aad5-4790-b768-765ee93ac9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.376884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4068-aad5-4790-b768-765ee93ac9cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.404299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:05.404299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.404299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.404299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.420693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:05.420693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:05.440188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:05.441137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:05.442051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:05.442051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:05.442051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:05.442051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:05.442051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 359/706] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:05.442051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.510436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf4068-aad5-4790-b768-765ee93ac9cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:05.511887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf4068-aad5-4790-b768-765ee93ac9cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:05.575417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcfe47-aa5d-4572-984d-82a6cd654d28 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.620227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:05.621058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.621343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.621618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.623467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:05.623868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:05.625439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386da20-75cd-4427-8634-1c3238b34645 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:05.633867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 348/707] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:32:05.635496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4833ece-86f2-4284-8c24-f222edcc4b71 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.636001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4833ece-86f2-4284-8c24-f222edcc4b71 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.636498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4833ece-86f2-4284-8c24-f222edcc4b71 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.638885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 349/708] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:05.643948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4833ece-86f2-4284-8c24-f222edcc4b71 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.644103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 360/709] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:05.645982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.646369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.646743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.667173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:05.669229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:05.747515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4833ece-86f2-4284-8c24-f222edcc4b71 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.760275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 361/710] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:05.761245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.783906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:05.784965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.785389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.785801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.796975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:05.798611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:05.817628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:05.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:05.820130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:05.820130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:05.820130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:05.820130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:05.822163 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 350/711] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:05.828355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:05.881393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50e381-63fc-4df7-a455-fccbf3e6d918 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.918877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:05.920208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:05.920630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:05.920961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:05.921738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:05.923234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:05.932194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2334a13-a2da-4c53-b003-16e5dd471df4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:05.952194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 362/712] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:32:05 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:05.952194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:06.012560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.013606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.013912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.014186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.016903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.019599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:06.054807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:06.055893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:06.056800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:06.059557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 351/713] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:05 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:06.081635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:06.176189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b4d0-5e04-4b91-b80d-3f7079d2a55d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:06.212656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64bf6d-4a5b-4449-b8b4-9e4d3acae2d2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:06.213058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.213723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.214013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.214292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.215017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:06.215327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.217847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.229734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 363/714] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:06.235786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:06.254894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.255806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.256530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.256811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.257794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] RBAC: Adding query filter params (name=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:06.258101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.259939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.268247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 352/715] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:06.274120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:06.372889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd85cda-b79d-4287-ac28-dcf95eaab04d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:06.407919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.408743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.409026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.409301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.410363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790c61c-03e6-4f97-a4cf-7b5895b4d5cb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:06.421719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.423166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.444944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.445529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.445811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.446084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.446796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:06.447100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.448512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:06.450188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:06.452697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:06.454801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 364/716] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:06.460550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 353/717] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:06.461521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:06.666403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0776519-ab06-4745-a577-484ac0603f64 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:06.712231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.712862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.713151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.713428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.714227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:06.714537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.715964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa674241-efb9-40ee-8f2d-5381428ee84a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.733838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 365/718] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:06.739450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:06.932397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd220b-c848-4b31-af93-3da12b744fb7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:06.976377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:06.976962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:06.977242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:06.977514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:06.978355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=62c3b98740d14ab49d91783086150e88) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:06.978664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:06.980134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c24c7-7afd-4c89-902a-29546fe757dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:06.987956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 354/719] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:06 2022] GET /identity/v3/role_assignments?user.id=62c3b98740d14ab49d91783086150e88&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:07.278620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de1fae-24f2-4239-8d17-4c72252c059e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.279572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de1fae-24f2-4239-8d17-4c72252c059e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.280303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de1fae-24f2-4239-8d17-4c72252c059e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.281636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 366/720] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:07.287164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.287623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.287700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.299580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a91ad9-5144-469e-b8e0-3a2cdec9817c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.299859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a91ad9-5144-469e-b8e0-3a2cdec9817c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.300136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a91ad9-5144-469e-b8e0-3a2cdec9817c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.301816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 367/721] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:07.312532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.312826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.313093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.331527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:07.332077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:07.338144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:07.339875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:07.449847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 355/722] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:07.487541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:07.505467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 368/723] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 12 17:32:07.530990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:07.614288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb8985-6114-4276-99c2-1bf5f8f6674b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:07.659099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:07.659901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.660190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.660479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.680722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:07.691357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:07.692496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:07.693367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 356/724] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:07.698178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:07.702044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cccee1-7af9-468c-8e59-72e821e7a711 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:07.780368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:07.781088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.781316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.781588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.782331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:07.783696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:07.820642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 369/725] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:07 2022] POST /identity/v3/services => generated 251 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:32:07.856099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec53e-7620-4f4f-9172-7abf8ecd0457 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:07.917279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:07.918333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:07.918799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:07.919248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:07.920814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:07.921379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:07.922972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:07.940827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 357/726] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:07.946479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:08.099341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e346a1-2068-44fa-accb-512a17508ede None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:08.133380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:08.134163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:08.134442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:08.134714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:08.145824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:08.147249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:08.164462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:08.165400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:08.166286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:08.166286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:08.166286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:08.166286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:08.166286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 370/727] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:08.171137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:08.270302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7841-9487-4ffc-80cc-be5c19559f75 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:08.309618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:08.310405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:08.310683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:08.310953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:08.311663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:08.311960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:08.313411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:08.328557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 358/728] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:08.333509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:08.458373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce7c6a-a311-4931-aa32-7f0169b5274a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:08.509810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:08.510621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:08.510909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:08.511186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:08.522305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:08.523744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:08.541869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:08.542853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:08.543774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 371/729] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:08.550105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:08.726661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38e4b-70ad-4d87-9b21-6e9cee07a520 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:08.780866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:08.781548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:08.781832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:08.782110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:08.782952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:08.783260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:08.784711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f7aec-ebb7-446f-83b2-8fda7910ecae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:08.805811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 359/730] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 12 17:32:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:08.811519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:08.949565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8cd1d-2e4e-498c-9004-0cbade353297 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:08.997885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:08.997885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:08.997885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:08.997885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.087329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:09.092893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:09.129801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47d2f7-31ad-4c88-9dc7-39c28b28760e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.130129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47d2f7-31ad-4c88-9dc7-39c28b28760e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.130360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47d2f7-31ad-4c88-9dc7-39c28b28760e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.132010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 360/731] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:09.141220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.141504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.141787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.181208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:09.182414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:09.194207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 372/732] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 12 17:32:09.201291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:09.316479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 361/733] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:09.356993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:09.383994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9ee4b-0593-491a-a844-605877e5c4f2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:09.457510 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:09.458576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.459057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.459546 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.474076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:09.475987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:09.497340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:09.499005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:09.499952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:09.499952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:09.499952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:09.499952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:09.503158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 373/734] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:09.509508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:09.555987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ca083-a529-4793-8cf2-d87a1c95003b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:09.610724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e924-b472-4a0f-8423-def43eff5c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:09.614704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e924-b472-4a0f-8423-def43eff5c67 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.614704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e924-b472-4a0f-8423-def43eff5c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.614704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e924-b472-4a0f-8423-def43eff5c67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.629091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8e924-b472-4a0f-8423-def43eff5c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-77f8e924-b472-4a0f-8423-def43eff5c67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:09.636963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:09.637929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:09.639348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:09.643217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 362/735] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:32:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:09.644669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1f21d-be41-4f42-b00b-4ae359725a55 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.644959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1f21d-be41-4f42-b00b-4ae359725a55 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.645236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1f21d-be41-4f42-b00b-4ae359725a55 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.646661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 363/736] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:09.650576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fa21e-c576-4c64-98b2-bac4572e8c60 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.650576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fa21e-c576-4c64-98b2-bac4572e8c60 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.650576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fa21e-c576-4c64-98b2-bac4572e8c60 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.653275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 364/737] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:09.654351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fa21e-c576-4c64-98b2-bac4572e8c60 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:09.673035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84396096-2e62-4625-aebb-53cf1e4bb4e8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:09.710635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:09.711322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.711586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.711861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.712605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:09.712915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:09.714269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc58b9a-dbfe-4a38-8905-a930e374f5f1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:09.731182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 374/738] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:09.731182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.731182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.731182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.745880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:09.747057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:09.757329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fa21e-c576-4c64-98b2-bac4572e8c60 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:09.790231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:09.790890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.791170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.791533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.792196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:09.792499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:09.793841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f4d59-196b-439c-b722-6cc296d3af12 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:09.807071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 365/739] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:32:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:09.807071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.807071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.807071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.828348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 375/740] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:09.828528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:09.828622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:09.828731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:09.887780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 366/741] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 12 17:32:09.889358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:09.934662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75fff4-738f-4f2c-ad38-27f4dc365360 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:09.968478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:09.969275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:09.969555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:09.969885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:09.976541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:09.977968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:09.991515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f5514-6843-4f9a-8340-615f87f20c84 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:09.991671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:09.992573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:09.993507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:09.997601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 376/742] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:09.998145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.026928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.027706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.027981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.028275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.038675 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.040135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:10.057135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:10.058047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:10.058985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:10.058985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:10.058985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:10.058985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:10.058985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 367/743] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:10.059849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.102285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406e251-9efe-466d-be6a-52a281635e35 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.135423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.136080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.136383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.136654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.137366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.139479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.152936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 377/744] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:09 2022] POST /identity/v3/services => generated 240 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:10.153561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.168750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9e488-6930-4952-9956-93dde3d43504 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.223779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.225294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.225977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.226924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.246803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:10.256446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:10.257475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:10.258319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 368/745] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:10.259034 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.290961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790be7-0eda-446d-8fc9-c72bbd4ad6c1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.338416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.339958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.340420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.340885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.342236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:10.342746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.345047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.361314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 378/746] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:10.362177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.377208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3b81-8a04-4bd2-81ca-c6b9de4f98ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.413913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.414575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.414852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.415122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.415841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:10.416140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.417520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.430350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 369/747] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:10.431174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.465399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d4e06-d5ca-4dc8-a813-81da02f0efad None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.503001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.503671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.503953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.504260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.505013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:10.505316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.506671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.516128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 379/748] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:10.516978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.537730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bfcb0-7163-4b58-9fb4-0b453a42cd63 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.571252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.571905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.572205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.572495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.573318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=972f23cd94bf4051825701d2aaddfe6d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:10.573623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.575023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.581495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 370/749] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/role_assignments?user.id=972f23cd94bf4051825701d2aaddfe6d&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:10.584896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.656930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e121-7413-42f7-8648-9486648e56eb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.694875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.695614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.695890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.696184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.704853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.706343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.715249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c5231-c8ef-426f-a205-f9420aefcb67 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0a92226b-fdc4-4bb0-99da-d80751cf825e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:10.722058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:10.723081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:10.724895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 380/750] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:10.726261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3449d2c-7673-4ba4-93a8-4429907593bf None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.727971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3449d2c-7673-4ba4-93a8-4429907593bf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.727971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3449d2c-7673-4ba4-93a8-4429907593bf None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.730074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 381/751] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:10.730074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3449d2c-7673-4ba4-93a8-4429907593bf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.758053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.758845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.759121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.759387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.772884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.774214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a35a2a2f-2938-492b-be9a-f0709a8d9f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:10.792870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:10.793846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:10.794762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:10.794762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:10.794762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:10.794762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:10.794762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 371/752] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:10.798702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.798702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.798702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.817725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:10.819277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:10.856933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3449d2c-7673-4ba4-93a8-4429907593bf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:10.892416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 372/753] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:10.893428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:10.898480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:10.899378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:10.899846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:10.900283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:10.901400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:10.901850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:10.904583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:10.925263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 382/754] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:10.930817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.021893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296a7b8-a27f-4a4d-9829-ae8ef28bd278 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.049318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8194846-b17e-4116-bfac-7ede0742bdaf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.056400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.057329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.057329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.057627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.058350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:11.058652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.060657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.076298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 373/755] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:11.077187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.085785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.086672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.087096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.087511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.154542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.156364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.184516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad220f60-0788-47b6-8047-ddcda950e085 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.218493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.219526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.219943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.220403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.221186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.222664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:11.238958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:11.240018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:11.241126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 374/756] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:11.241985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.277378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 383/757] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:11.278506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.337873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0bc7-893f-4cc2-ad50-880e11a4e8dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.371672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.373081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.373373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.373669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.379091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a51ab-115f-42b9-9a11-9befb42437e2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.382737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.383676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:11.398992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:11.400009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:11.400982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:11.405362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 375/758] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:11.406386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.420651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.421584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.422037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.422506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.423430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:11.423907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.425665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.430476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 384/759] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:11.431588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.505230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce765ee-30ad-4956-8a5e-75db51b465bf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.532102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36109f-6a91-4a21-9ba0-2bc795dbbf25 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.538535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.539323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.539600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.539867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.553257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.555000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.566555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.567463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.567926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.568416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.569418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:11.569916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.571475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:11.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:11.576314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:11.577359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:11.577359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:11.577359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:11.577359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:11.584234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 376/760] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:11.585347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.596795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 385/761] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:11.597887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.710513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d1f2d-7588-40e7-b885-e55c6f87b20f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.728417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabee80-d9c0-4fac-9923-21be59b92f8a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.744571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.745321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.745674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.746017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.746786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:11.747534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.749342 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.754410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 377/762] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:11.755367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.772201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.773327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.773749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.774313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.775470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:11.775944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.777488 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.796286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 386/763] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:11.796286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.878440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ada7-2a38-4415-b29d-074ff6e264ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.910835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00022-cdac-47dd-81a6-b91f4fc978f9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:11.912677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.914059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.914059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.914059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.925234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.926775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:11.945212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:11.946331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:11.947370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:11.947370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:11.947370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:11.947370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:11.948256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 378/764] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:11.949188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:11.952316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:11.953153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:11.953631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:11.954073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:11.955081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] RBAC: Adding query filter params (interface=public, service_id=789758c4df1242c1989b62ee42c699cc, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:11.955610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:11.957227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:11.966511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 387/765] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/endpoints?service_id=789758c4df1242c1989b62ee42c699cc&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:11.968668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.055333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff13096-f41b-46db-b469-1ed1358863d8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.072527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f8e3-3e45-453a-832e-4d38907850ba None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.089416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.090210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.090493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.090770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.097487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.098922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.107324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.108235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.108741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.109183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.110116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:12.110627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.112137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:12.113689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:12.114555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:12.115425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:12.119762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 379/766] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:12.120632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.128837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 388/767] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:12.129897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.225811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae86ec-c90b-4de3-85e1-0db6ea04e7e1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.232415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5283a6-463a-4161-83d2-c5dd4016335f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.262416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.263344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.263790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.264293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.265104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.266588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.266926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.267581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.267862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.268222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.269121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:12.269281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.270688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.279558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 380/768] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:12.280824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.283102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 389/769] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:12.289005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.397308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830f02-d304-4733-9d29-769f3bbf5aff None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.432689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.433364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.433647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.433923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.434772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3a892d7213414aa589db028d1b7c27c2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:12.435671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.437319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854776c8-cb60-4032-9150-8d0b047bbd4b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.443636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 390/770] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/role_assignments?user.id=3a892d7213414aa589db028d1b7c27c2&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:12.457401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334fef6-274f-4492-869f-ae8dff278852 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.519892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbb67-0af7-4870-be2b-5e3a6f16161c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.519892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbb67-0af7-4870-be2b-5e3a6f16161c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.519892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcbb67-0af7-4870-be2b-5e3a6f16161c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.521502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 391/771] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:12.527006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895b918-7310-470c-a480-289ad5605f92 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.527295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895b918-7310-470c-a480-289ad5605f92 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.527575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895b918-7310-470c-a480-289ad5605f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.555947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.556793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.557076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.557354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.558963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e895b918-7310-470c-a480-289ad5605f92 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:12.560860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e895b918-7310-470c-a480-289ad5605f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:12.570950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.572473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:12.600471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:12.601437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:12.602322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:12.603496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 381/772] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:12.609756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.687297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 392/773] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:12.712763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895b918-7310-470c-a480-289ad5605f92 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.826705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895b918-7310-470c-a480-289ad5605f92 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.850536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c7c41-143b-4727-8cc4-30b266ff9dfe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:12.874459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.875629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.876131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.876637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] PATH_INFO: `/v3/services/network` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.877485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.879037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.885697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:12.886422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:12.886700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:12.886972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:12.887808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:12.888826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:12.892185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:12.907378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:12.908426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:12.909257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:12.909980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 382/774] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:12.909980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:12.916835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 393/775] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:12.918354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:13.008090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7716-7c3a-4cc9-9e8b-2c30a0a629d1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.055978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e68976-944f-4f48-b4e6-51dd1aa4e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.057487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e68976-944f-4f48-b4e6-51dd1aa4e569 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.057963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e68976-944f-4f48-b4e6-51dd1aa4e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.058436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e68976-944f-4f48-b4e6-51dd1aa4e569 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/62c3b98740d14ab49d91783086150e88/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.062594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e68976-944f-4f48-b4e6-51dd1aa4e569 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=62c3b98740d14ab49d91783086150e88, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:13.065930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e68976-944f-4f48-b4e6-51dd1aa4e569 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:13.091898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 383/776] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:32:12 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/62c3b98740d14ab49d91783086150e88/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:32:13.094445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cf1e7-8547-4f2f-95c6-4ea446902044 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.095154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cf1e7-8547-4f2f-95c6-4ea446902044 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.095619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cf1e7-8547-4f2f-95c6-4ea446902044 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.097643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 384/777] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:13.108441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.108747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.109009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.123227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7afb2-6102-48bb-8793-ed62ad266a16 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.127467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:13.128697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:13.168347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.169126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.169700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.170073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.170799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:13.171106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:13.172687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:13.180279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 394/778] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:13.186206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:13.212044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 385/779] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:13.267597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:13.328718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebad8-3c6d-42d7-ae83-209bf390938a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.370537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.371323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.371722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.371895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.372717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] RBAC: Adding query filter params (type=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:13.372717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:13.374431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:13.380648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 395/780] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:13.386324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:13.479999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7b42-b815-41d8-940d-c93087fd7852 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.504693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2295d-babe-4bfe-9f71-d075e9ee2b1c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.540180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.540969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.541250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.541517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.541934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.542865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.543361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.543835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.545710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] RBAC: Adding query filter params (interface=public, service_id=b2501dfe55da48509fc206470c05bacf, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:13.546238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:13.547803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:13.558116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 396/781] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:13 2022] GET /identity/v3/endpoints?service_id=b2501dfe55da48509fc206470c05bacf&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:13.559644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:13.564949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:13.580366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:13.581382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:13.582857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 386/782] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:13.587367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:13.683609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f8ca5-71d5-4d7e-a52c-0dae6bedcbbb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.719858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.720927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.721433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.721910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.722809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:13.724387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:13.736301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 397/783] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:13 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:13.798021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa280f-e24f-4663-8563-b2b4b97cd607 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:13.850379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:13.851054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:13.851332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:13.851598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:13.852373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:13.852676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:13.854110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:13.868258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 387/784] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:13.875107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:14.026601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54311223-7f06-4689-b33d-bb88fbd5ea7e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:14.062354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:14.063065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:14.063344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:14.063614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:14.074352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:14.076359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:14.093685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:14.094662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:14.096000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:14.096000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:14.096000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:14.096000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:14.096000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 398/785] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:14.100423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:14.213145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcd5b3-21ad-4c76-a97d-4c2bf7426db3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:14.256599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:14.257265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:14.257542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:14.257817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:14.258538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:14.258843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:14.260273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:14.276914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 388/786] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:14.282866 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:14.411146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72572170-6e23-4a7a-80cb-dbb862241390 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:14.448317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:14.449113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:14.449397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:14.449673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:14.461073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:14.462496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:14.482714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:14.483734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:14.485354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:14.488266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 399/787] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:14.489463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:14.613940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799639c3-9bd8-4ff7-815a-58a8d9677b01 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:14.656623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:14.657292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:14.657568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:14.657838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:14.658682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:14.658987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:14.660421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:14.678286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 389/788] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 12 17:32:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:14.685554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:14.807603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327bdb54-560c-4ecc-8c45-7ed97299a377 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:14.850361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:14.851156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:14.851442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:14.851718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:14.867884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:14.869367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:14.891194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:14.892615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:14.893521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:14.893521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:14.893521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:14.893521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:14.896795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 400/789] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:14.902328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.085172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87187e9-dce2-46fc-b8ae-3ed990dcef20 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.118930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.119586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.119867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.120137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.120882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:15.121181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:15.123258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7b6c7-c0c3-448a-8be7-f159bc24afbc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:15.134983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 390/790] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:15.141244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.258383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5e278-a27b-49ea-a32b-35ae831f4ce0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.258711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5e278-a27b-49ea-a32b-35ae831f4ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.258946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5e278-a27b-49ea-a32b-35ae831f4ce0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.260645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 391/791] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:15.266125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32069f22-4409-4cd0-93ee-9461e0ea867e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.266406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32069f22-4409-4cd0-93ee-9461e0ea867e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.266680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32069f22-4409-4cd0-93ee-9461e0ea867e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.288760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-32069f22-4409-4cd0-93ee-9461e0ea867e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:15.289937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32069f22-4409-4cd0-93ee-9461e0ea867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:15.296602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538a9c8-bf41-4d96-973e-36ece62d5a63 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.331945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.332908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.333284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.333659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.340584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:15.342721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9af51a7f-8868-40bd-a2c1-7a9f03dc2ae1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:15.358146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:15.359246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:15.360383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:15.362118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 401/792] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:15.367603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d42e-08e1-4c79-a465-9bbd602edf2b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.367603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d42e-08e1-4c79-a465-9bbd602edf2b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.367603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d42e-08e1-4c79-a465-9bbd602edf2b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.367603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 402/793] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:15.373856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d42e-08e1-4c79-a465-9bbd602edf2b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.375517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 392/794] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:15.377327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.378648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.379160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.398922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:15.401118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:15.481892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 393/795] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:15.482649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d42e-08e1-4c79-a465-9bbd602edf2b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.482763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.529253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.529918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.530198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.530472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.531262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:15.531565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:15.533030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:15.544916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 403/796] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:15.545755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.609919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee03df3-15a2-4c74-827f-e393a4d54bc2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.651051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.651857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.652137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.652457 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.665035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:15.672991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:15.673993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:15.675258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:15.679395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 394/797] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:15.680229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.682298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef27037-cc18-4e60-ae8c-392623fbc3bc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.720681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.721923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.722443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.722926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.723812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:15.725494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:15.739404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:15.740554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 12 17:32:15.741657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:15.750417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 404/798] 10.222.0.73 () {56 vars in 1129 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:15.751648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.787360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbbca1-24f8-4aca-98ce-322d7edf3b76 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.822806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.823814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.823999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.824337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/972f23cd94bf4051825701d2aaddfe6d/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.827664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=972f23cd94bf4051825701d2aaddfe6d, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:15.829123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:15.844826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 395/799] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:32:15 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/972f23cd94bf4051825701d2aaddfe6d/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:32:15.845811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a26c-7f8f-47c4-bf26-54738d628e62 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:15.921020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:15.921020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.921020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.921020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.921883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:15.921883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:15.921883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bcafb-6726-4501-90e7-3c95ee8bffe2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:15.930237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 405/800] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:15.931670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab8631-6aeb-4ab0-bf0f-e2b39f8bd0a2 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:15.931883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab8631-6aeb-4ab0-bf0f-e2b39f8bd0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:15.937062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab8631-6aeb-4ab0-bf0f-e2b39f8bd0a2 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:15.942653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 406/801] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:15.942653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab8631-6aeb-4ab0-bf0f-e2b39f8bd0a2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:15.984890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3b786-d5ba-41b7-9883-9adfbd550bd4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.027933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.029786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.030906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.031409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.032730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:16.033440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.035941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022fb91-573d-4bee-a965-78120940d514 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.042461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 396/802] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:16.044816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.045299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.045929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.070955 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab8631-6aeb-4ab0-bf0f-e2b39f8bd0a2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.071743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:16.073169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:16.111956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.112919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.114005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.114295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.126281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.128405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.148552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 397/803] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:16.149638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:16.149817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:16.150680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:16.151550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:16.151550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:16.151550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:16.151550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:16.151550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 407/804] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:16.157607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.268242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa77507-1cdb-4fff-b5df-21c1c6833f78 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.269097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd18d58-a664-43ac-9427-016934c0d514 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.303849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.304067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.304768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.305050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.305159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.305319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.305765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.306028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:16.306320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.306472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.307334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:16.307674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.307923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.309416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.313069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 398/805] 10.222.0.73 () {56 vars in 1124 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:16.313776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.322055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 408/806] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:16.323057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.419103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f5528-2395-404a-92e0-a33755732024 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.428487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9d07e-9876-4281-9728-172fe6abd26b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.455880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.457028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.457311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.457586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.463370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.464245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.464683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.465096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.465960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.467516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.468622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:16.477912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:16.479648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 399/807] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:16.485282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.502688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 409/808] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:16.504573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.612918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dcd2d-870f-4093-89b3-d84ec03df579 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.626016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaa0bf-a08f-4303-9cb0-a69668e122bd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.661903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.663347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.663347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.663347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.664543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.665423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.665875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.666315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.667275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:16.667773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.670739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.673074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.674920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.681317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 410/809] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:16.687831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:16.693121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:16.694083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:16.695029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:16.696442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 400/810] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:16.705288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.827202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8e126-8c77-4633-9f0f-c68b0f906dc7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.876098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.876668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.877135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.877386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.878254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:16.878557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.880100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.894531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f54c2-cc18-4a00-ac28-4f12aab0f8da None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:16.908201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 401/811] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:16.912082 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:16.940613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:16.941333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:16.941617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:16.941913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:16.955518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:16.958038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:16.978492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:16.979467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:16.980396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:16.980396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:16.980396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:16.980396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:16.980396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 411/812] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:16.988700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.047833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8d4ce-97a1-4949-a7f1-5eaf5d9422c1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.088822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.089719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.090002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.090269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.108468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.110518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.116002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399382e-67c6-49cb-b914-8ad3641dcaab None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:17.136670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:17.137627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:17.138547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:17.138547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:17.138547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:17.138547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:17.139866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 402/813] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:17.149810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.159954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.160674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.161170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.161675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.162564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:17.163117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.164900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc218b-5f0a-406f-aa22-d2bb745a4cd3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.178858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 412/814] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:17.180269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b54d75-ac08-4694-82d5-3fb88ab84b8b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.180564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b54d75-ac08-4694-82d5-3fb88ab84b8b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.180839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b54d75-ac08-4694-82d5-3fb88ab84b8b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.182301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 413/815] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:17.188812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b54d75-ac08-4694-82d5-3fb88ab84b8b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.266463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698acae-6e16-4c3e-ab06-5c7d7bbe7869 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.301433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7945e-3250-4107-9220-165346c1b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.302339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7945e-3250-4107-9220-165346c1b5df None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.302825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7945e-3250-4107-9220-165346c1b5df None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.303240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7945e-3250-4107-9220-165346c1b5df None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.304104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7945e-3250-4107-9220-165346c1b5df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:17.304633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7945e-3250-4107-9220-165346c1b5df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.306456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7945e-3250-4107-9220-165346c1b5df None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.312829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b54d75-ac08-4694-82d5-3fb88ab84b8b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.326404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 403/816] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:17.328039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.328585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.329026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.346193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:17.347809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:17.352031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.354467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.354853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.355240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.364508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.368358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:17.392841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:17.393940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:17.394873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 414/817] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:17.395535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.436263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 404/818] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:17.437682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.536610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f0bd1-5e50-49c6-a53e-71c701bf1cfd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.556491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a05ef-3a62-44cd-832c-fa55b942902d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.574471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.576200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.576605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.577101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.587348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.589030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.602611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.603526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.603984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.604411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.605390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:17.605871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:17.608335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:17.610036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:17.611853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:17.613050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.621602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 415/819] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:17.622790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.641893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 405/820] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:17.643089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.756679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8725be7-9fda-4851-bff1-9ce41dd231bd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.762364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b0bc4-19f6-4ec2-abaa-d61e8d52fe29 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.797601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.798496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.798940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.799364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.800553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:17.801095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.803083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.803376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.804328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.804713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.805087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] PATH_INFO: `/v3/services/network` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.805853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:17.807305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:17.818659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 406/821] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:17.819754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:17.829657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:17.830898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 12 17:32:17.831728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:17.834955 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 416/822] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:17.835769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:17.926450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad01eba-b7fb-40a2-b8b1-ad081181fe17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:17.959450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:17.960438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:17.960926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:17.961374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:17.970854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bc782-0783-430f-88a6-b5d036455e54 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.011124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.012126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.012505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.012853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/3a892d7213414aa589db028d1b7c27c2/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.031937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.033745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.033881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=3a892d7213414aa589db028d1b7c27c2, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.037225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.055332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 417/823] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:32:17 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/3a892d7213414aa589db028d1b7c27c2/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:32:18.061895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.144235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 407/824] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:18.153254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.219986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cae946-8ff6-4594-9877-e3c645d9038d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.257400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.258199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.258822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.258822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.259657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:18.260091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.261658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3cc6f-7e8d-4449-9a9f-a94ead8b0cc6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.268190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 418/825] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:18.273536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf90ce-c10e-416d-bc43-ddd755f13ea7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.273935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf90ce-c10e-416d-bc43-ddd755f13ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.274381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf90ce-c10e-416d-bc43-ddd755f13ea7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.276231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 419/826] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:18.277362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf90ce-c10e-416d-bc43-ddd755f13ea7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.319765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2273d-c6f2-4a06-ab98-4bc30e5b0cac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.364611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.366026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.366303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.366576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.379077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.380545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.391710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf90ce-c10e-416d-bc43-ddd755f13ea7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1eefee73-d012-4c5e-9485-b70dfeb6f7dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:18.399164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:18.400058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:18.400990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:18.400990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:18.400990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:18.400990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:18.400990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 408/827] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:18.401698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.401859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.402433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.420240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:18.421530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:18.430737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.431410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.431685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.431965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.432708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] RBAC: Adding query filter params (type=network) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:18.433016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.434378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.440054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 420/828] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:18.440915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.493520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 409/829] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:18.496328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.589405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a855c96-36f5-48b7-b131-83380fcb6b49 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.622085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb1357-4ac2-47bf-b9ec-f978f875a52c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.631394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.632426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.632912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.633384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.634396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:18.634902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.636497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.652607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 421/830] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:18.654414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.660010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.660708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.660984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.661253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.661981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.663322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.691031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 410/831] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:18.692141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.787636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f358a90-68a7-4d74-8499-1ab814c0328a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.858797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7908-0f03-4aef-b15b-f23918b98000 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:18.865023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.866099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.866387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.866665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.891535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:18.906382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:18.907543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:18.909564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:18.910083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:18.911816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:18.912769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:18.913608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:18.917649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 422/832] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:18.924303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:18.931008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:18.936113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:18.960509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:18.961443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:18.962333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:18.965353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 411/833] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:18.971719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.066770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dee64-9e9b-4528-8868-d4ecebb69cbd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.101176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324c609-b3a6-4ed4-a676-a48267e96663 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.111188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.111898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.112288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.112637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.113745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:19.113745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.115325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.127020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 423/834] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:19.132314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.143884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.144626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.144908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.145181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.145992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:19.146302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.147777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.161144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 412/835] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:19.170141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.246402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faaff5-3a48-4ead-b570-07cbec79e470 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.280192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.281073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac3c6b-f94f-4cb2-924e-6be83e1c02db None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.281423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.281854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.282273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.292887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.295161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:19.312265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:19.313477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:19.314634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:19.314634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:19.314634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:19.314634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:19.316689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.316887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 424/836] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:19.317452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.317801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.318161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.319049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=62c3b98740d14ab49d91783086150e88) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:19.319425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.320996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a456b-834a-48eb-9e8a-177f4211ed08 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.322029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.332854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 413/837] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/role_assignments?user.id=62c3b98740d14ab49d91783086150e88&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:19.352851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eacd1-3f38-4264-a2bd-08ae8ec82cf0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.353169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eacd1-3f38-4264-a2bd-08ae8ec82cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.353417 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eacd1-3f38-4264-a2bd-08ae8ec82cf0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.355102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 414/838] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:19.361922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.362118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.362387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.381740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:19.382965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:19.434807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc658674-428f-4928-a141-0f6d6ced10cd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.470892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.471775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 415/839] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:19.472292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.472800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.473258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.474233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:19.474786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.476404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.491189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 425/840] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:19.497122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.497440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.614713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1e5e6-a566-4ee3-8da8-270d6bc6a50b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.631369 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614395d1-bd9c-4473-ab08-18fb75ff6778 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.657822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.658951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.659239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.659524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.668650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.669889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.671022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.671477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.671940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.672414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.673252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.674768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:19.686793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:19.687828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:19.688727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:19.689469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:19.690359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:19.691306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 426/841] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:19.693975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.694781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 416/842] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:19.705209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.797014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6fd1-54bf-41f2-a928-94c19db3aa2f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.832639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.833682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.834179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.834650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.835707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:19.835894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf689f59-571e-4cb6-9819-3a583578cb7c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:19.836446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.838075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.858056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 427/843] 10.222.0.73 () {56 vars in 1135 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:19.863649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.881835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:19.882832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:19.883367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:19.883843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:19.884852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:19.885384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:19.886994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:19.892054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 417/844] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:19.898273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:19.978887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72d6db-135f-44d5-85be-6e5b9e1f0eee None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.005831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b00e90-696d-4d00-b6cd-dd4c5ef508de None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.013749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.014765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.015141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.015490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.048453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.049576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.050815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.051272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.052190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:20.052692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:20.054236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:20.061173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 418/845] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:20.096723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:20.098928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:20.208980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 428/846] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:20.217299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:20.321563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3a60-5313-45ce-9b77-21c5188d7881 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.354210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.355005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.355283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.355551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.369516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:20.370960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:20.388338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:20.389466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:20.390360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:20.390360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:20.390360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:20.390360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:20.390360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 419/847] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:20.396734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:20.503875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a261-d6cf-45f5-9257-887ed1814dfd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.538874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.539532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.539816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.540092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.540844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:20.541147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:20.542493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7538b-c244-4e52-b0e3-4f0856bf3a13 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:20.554253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 429/848] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:20.559776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:20.650879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32d98-e683-477a-8524-ff35f4274d9d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.651208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32d98-e683-477a-8524-ff35f4274d9d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.651430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32d98-e683-477a-8524-ff35f4274d9d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.653060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 430/849] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:20.661244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e9f88-3638-49be-955d-af0727115a12 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.662018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e9f88-3638-49be-955d-af0727115a12 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.662531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e9f88-3638-49be-955d-af0727115a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.674331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf60e19-1d44-4fe5-bb38-ef04878b4985 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.681711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-963e9f88-3638-49be-955d-af0727115a12 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:20.682886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-963e9f88-3638-49be-955d-af0727115a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:20.715629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.717191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.717561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.718019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.726491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:20.728596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:20.749356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:20.750307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:20.751258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 420/850] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:20.755212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 431/851] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:20.761252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:20.774650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e9f88-3638-49be-955d-af0727115a12 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:20.878926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb932d0-d512-4ca8-8fe6-eea4defa0d07 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.893736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e9f88-3638-49be-955d-af0727115a12 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:20.921060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.922538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.923312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.923808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.924825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:20.925334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:20.926932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:20.927333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:20.927719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:20.927997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:20.928297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:20.939236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:20.947717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:20.948704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:32:20.949597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:20.952451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 421/853] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:20.952565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 432/853] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:20.953317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:20.957298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:21.070419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596fdb3-5a74-4d57-addc-c431b4a34fdc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:21.078319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99700c2e-f6b0-4acf-bf17-ee092570c9d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:21.112108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:21.112814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.113088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.113358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.113944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:21.114408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:21.114408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:21.115350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.116015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:21.116327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.116799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.118362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=972f23cd94bf4051825701d2aaddfe6d) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:21.118896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:21.120518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:21.132398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 422/854] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:21.132398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:21.133293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 433/855] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:20 2022] GET /identity/v3/role_assignments?user.id=972f23cd94bf4051825701d2aaddfe6d&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:21.248341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3690d55-34ec-49e7-b69f-bba59663fcd9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:21.287005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:21.287799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.288071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.288360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.300019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:21.301552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:21.320525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:21.321461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:21.322379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:21.322379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:21.322379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:21.322379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:21.326495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 423/856] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:21.327907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:21.445321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded41b2-2bea-4fd2-923b-0f6b2a13cab8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:21.492079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:21.492766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.493050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.493300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.494019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:21.494322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:21.496850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:21.516295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 434/857] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:21.518263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:21.645141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef6ef-bca5-4ada-815c-ec4917696547 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:21.680554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:21.681366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.681649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.681924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.690638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:21.692082 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-80c345bd-39f4-49dc-a602-4c7ea93ace73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:21.707643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:21.708671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:21.709630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:21.713536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 424/858] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:32:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:21.714891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:21.820293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621deb9-d0ca-4e50-b89d-702496e0e9b8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.820613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621deb9-d0ca-4e50-b89d-702496e0e9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.820844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621deb9-d0ca-4e50-b89d-702496e0e9b8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.822441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 425/859] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:21.826320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260f16-2f14-4740-b9ff-c8711ffd4209 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:21.828224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.828517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.828794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.848798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:21.849986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:21.877334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:21.878596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:21.879224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:21.880025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:21.881435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:21.882008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:21.883789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:21.910199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 435/860] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 12 17:32:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:21.912484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:21.932288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 426/861] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:21.947065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.047216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2dd85-9480-46ba-86fd-cf00b3278c3f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.061296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864284b1-dd9c-4b5c-8414-9fbe8a7b329e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.083639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.084679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.085134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.085579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.100889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.101779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.102163 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.102551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.103314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.104801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:22.117316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:22.118285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:22.119122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:22.123565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 427/862] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 12 17:32:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:22.124540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.164953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.167072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.223595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9997465-4d97-4fbe-9a54-ce833a9a8c47 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.260322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.260718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.260907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.261297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.262096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:22.262401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.264488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.274426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 428/863] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:22.275386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.283387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 436/864] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:22.284978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.378910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bf0f2-292c-4769-8474-e7e6d6a4cadf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.388013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192fee0-4311-4337-8ac7-8f2bf42386c7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.415963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.416560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.416811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.417086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.417791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:22.418096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.420915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab7af5-75a5-4e68-a874-85cb780f99b1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.429366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.430613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.431133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.431637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.432670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 429/865] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:22.440766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aaf3e-2e21-40e6-a0cb-abe9ae839508 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.441111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aaf3e-2e21-40e6-a0cb-abe9ae839508 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.441322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aaf3e-2e21-40e6-a0cb-abe9ae839508 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.447140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 430/866] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:22.447364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.448604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.448889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.449164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.449824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.467422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:22.468732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:22.472316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:22.473263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:22.474147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:22.474147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:22.474147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:22.474147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:22.478298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 437/867] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:22.479702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.551359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 431/868] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:22.572576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.623972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be843a-f8dc-40c6-b4d2-3802e2595958 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.663094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.664125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.664527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.664879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.665681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:22.666105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.667493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.691257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 438/869] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:22.699341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:22.700779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5d355-ab8e-4ebe-a001-d56ef5ca7e1d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.735094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.736081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.736574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.737019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.737923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.739450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.755622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 432/870] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:22 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:22.820192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f400e0-3a42-4142-b2e8-ec29263dbd19 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:22.863767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:22.864655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:22.864938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:22.865204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:22.873846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:22.875308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:22.891316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:22.892336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:22.893235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:22.897520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 439/871] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:22.898922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:23.005182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae498-f804-423e-8a99-92f32b5f54dd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:23.038731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:23.039392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:23.039670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:23.039938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:23.040767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:23.041071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:23.042491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7a008-2981-41e5-952f-63ac2b4dd605 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:23.055038 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 433/872] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:23.061385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:23.161147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18b0f0-5ec1-4dce-a709-99768f17cb79 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:23.195089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:23.195722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:23.196033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:23.196299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:23.197126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3a892d7213414aa589db028d1b7c27c2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:23.197435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:23.198840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae118b6e-0863-4013-b189-4fa43a3e1778 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:23.212352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 440/873] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:23 2022] GET /identity/v3/role_assignments?user.id=3a892d7213414aa589db028d1b7c27c2&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:23.599021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e72ab-0c63-4b08-a248-1601d962b27d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:23.599420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e72ab-0c63-4b08-a248-1601d962b27d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:23.599601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e72ab-0c63-4b08-a248-1601d962b27d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:23.601376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 434/874] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:23.607837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:23.608136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:23.608438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:23.629008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:23.630191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:23.697737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 441/875] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:23.720385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e72ab-0c63-4b08-a248-1601d962b27d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:23.822551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e72ab-0c63-4b08-a248-1601d962b27d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:23.857724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:23.858783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:23.859207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:23.859633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:23.860442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:23.861938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:23.874647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:23.875859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:23.876927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:23.879332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 435/876] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:23.885167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:23.982523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae425a-5ace-4a36-8b9c-fc6323d9eba0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:24.018700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:24.019377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:24.019663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:24.019940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:24.020684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:24.020997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:24.022354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf370eb-cf46-4a54-bd52-741816504e4a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:24.027184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 442/877] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:24.033412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:24.133285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274293-eaf3-43d8-8a06-df4204088ff8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:24.166519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:24.167187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:24.167468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:24.167741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:24.168484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:24.168792 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:24.170887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873aeadb-8c3c-4d01-a3ae-05ba517f7c56 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:24.175829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 436/878] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:24.744661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17260bed-20e2-40f5-b05f-c981bf8864e3 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:24.745202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17260bed-20e2-40f5-b05f-c981bf8864e3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:24.745202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17260bed-20e2-40f5-b05f-c981bf8864e3 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:24.746801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 443/879] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:24.752448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58b061-42d5-4d6d-aff0-7b4c722e5f05 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:24.752861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58b061-42d5-4d6d-aff0-7b4c722e5f05 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:24.753025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58b061-42d5-4d6d-aff0-7b4c722e5f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:24.770316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6d58b061-42d5-4d6d-aff0-7b4c722e5f05 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:24.771498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6d58b061-42d5-4d6d-aff0-7b4c722e5f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:24.838334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 437/880] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:24.860413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17260bed-20e2-40f5-b05f-c981bf8864e3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:24.967007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17260bed-20e2-40f5-b05f-c981bf8864e3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:25.013824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b77472-5c97-4168-96e2-6c3f181fdfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:25.014514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b77472-5c97-4168-96e2-6c3f181fdfc2 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.014779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b77472-5c97-4168-96e2-6c3f181fdfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.015075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b77472-5c97-4168-96e2-6c3f181fdfc2 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.015788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b77472-5c97-4168-96e2-6c3f181fdfc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:25.017262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b77472-5c97-4168-96e2-6c3f181fdfc2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:25.037308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 444/881] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:24 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:25.204212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3033b-75bf-433d-9b78-e4134d7c3901 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.204212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3033b-75bf-433d-9b78-e4134d7c3901 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.204212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3033b-75bf-433d-9b78-e4134d7c3901 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.205745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 438/882] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:25.212652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.212928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.213206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.230614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:25.232261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:25.329264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 445/883] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:25.357345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3033b-75bf-433d-9b78-e4134d7c3901 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:25.478610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3033b-75bf-433d-9b78-e4134d7c3901 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:25.514684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:25.515704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.516148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.516588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.517398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:25.518911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7a58e0ec-16d5-4a97-b630-b1e6acfde7ec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:25.531328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:25.532650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:25.533761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:25.540336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 439/884] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:25.542357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:25.646607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2e5d3-878b-4a52-89db-28fcfe5b406d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:25.675181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9914aa-aa5f-4b53-ad43-6229fe05a541 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.675503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9914aa-aa5f-4b53-ad43-6229fe05a541 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.675747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9914aa-aa5f-4b53-ad43-6229fe05a541 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.677435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 440/885] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:25.679873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:25.680569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.680857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.681260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.681883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:25.682190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:25.682298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.682579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.682850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.683544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:25.689198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 446/886] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:25.694208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:25.705356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:25.706662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:25.783653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 441/887] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:25.801860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b4143-0c98-4033-953b-f793505f80ee None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:25.804246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:25.837754 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:25.838446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.838729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.839004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.839714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:25.840018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:25.842406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:25.848200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 447/888] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:25.853250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:25.913973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba667d13-bb84-4208-a2b2-5ec876fcc985 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:25.949466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:25.951275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:25.951636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:25.951972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:25.956495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbae72-812c-41a9-b726-01c00c8603e2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:25.964642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:25.975777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:25.976937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:25.977806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:25.982841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 442/889] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 12 17:32:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:25.984126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.001272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.002387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.002766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.003163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.004376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] RBAC: Adding query filter params (interface=public, service_id=9bc0bd3f7e6d487896a07064dc1ab4ed, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:26.004758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.006630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.018336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 448/890] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:25 2022] GET /identity/v3/endpoints?service_id=9bc0bd3f7e6d487896a07064dc1ab4ed&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:26.019305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.091677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e5d4-89d1-4417-a1d0-0716f9d58ad8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.117885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ea2c-cc30-477b-b7c6-57086387efb8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.126399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.127339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.127819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.128318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.129250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:26.129737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.131310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.140096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 443/891] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:32:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:26.146169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.159037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.159703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.159987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.160294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.161337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.163012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.176187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 449/892] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:26 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:32:26.261351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09691c3-a402-40f7-bd77-bdb054c1bcf0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.301050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.301855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.302132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.302464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.315231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.316863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:26.337164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:26.338156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:26.339059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:26.339059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:26.339059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:26.339059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:26.339059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 444/893] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:26.349717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.456345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ddc4-ae9d-4995-99b0-dc3c55c72031 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.493493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.494542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.495027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.495557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.496534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:26.497076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.498667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58031a67-4360-4995-a669-48f2efd32390 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.512640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 450/894] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:26.520017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.529520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634634-a58f-4805-9ae7-13677002f7c6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.529894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634634-a58f-4805-9ae7-13677002f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.530083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634634-a58f-4805-9ae7-13677002f7c6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.533225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 451/895] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:26.539849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.540151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.540448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.561197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:26.562421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:26.646337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f324b1-46b7-425e-ba49-6bb86edb420d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.656435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 452/896] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:26.678717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.687923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.688991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.689801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.690240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.701378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.703489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:26.722585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:26.723784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:26.724975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:26.727821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 445/897] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:32:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:26.734073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:26.787741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ebb5d-efd2-402f-990a-0bbe51ee8f6e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.832095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.833647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.834150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.834626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.836015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.838004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.856387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 453/898] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:26 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:32:26.872758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20401ea8-1188-486f-b27a-5de95de09366 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:26.919854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:26.920960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:26.921406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:26.921875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:26.922953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:26.923427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:26.925190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:26.947651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 446/899] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 12 17:32:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:26.953996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.063447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821ba0-ad14-4d72-b2ee-55264bce1046 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.100222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.100890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.101170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.101446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.170211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.171821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.270478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 454/900] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:26 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:27.276504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.378950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed7fd4-d177-4dd1-87c5-992ac9bcee3e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.412586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.413803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.414302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.414770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.427197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.429044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9777dd03-14e4-4951-8201-ae821c8cb0e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:27.447553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:27.448519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:27.449401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:27.449401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:27.449401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:27.449401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:27.451738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 447/901] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:27.457048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.476092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27651f-41e1-422f-84af-df7a7c9c02e1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.476447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27651f-41e1-422f-84af-df7a7c9c02e1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.476682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27651f-41e1-422f-84af-df7a7c9c02e1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.478015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 448/902] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:27.486217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.486495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.486769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.507839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:27.510085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:27.563341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede45773-b0a9-4de0-a67d-0295635717e9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.599737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 449/903] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:27.607430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.608382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.608862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.609328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.610265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:27.610747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.612353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.629488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.631874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 455/904] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:27.638750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.727287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e702-8cb5-4df7-b2f5-93bbe1d1ee2d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.734888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb30154-f8ad-4ec6-a355-60712c8139fd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.760712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.761513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.761788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.762050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.762714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.764827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.769806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.770598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.770882 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.771157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:27.776775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:27.777702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:27.779556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:27.780509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.780601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 450/905] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:27.780848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.787005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:27.793645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:27.794621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:27.795518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:27.800110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 456/906] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:27.801170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.912560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70fa3b-3b33-4d42-a482-5860567ea77a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.913182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464a825-c47e-4bb4-9c7a-f64ac07205b8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:27.951130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.951792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.952075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.952837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.953629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:27.953939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.955356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.960056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:27.961005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:27.961485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:27.962419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:27.963334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:27.964287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:27.966810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:27.972274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 457/907] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:27.973264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:27.978900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 451/908] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:27.984508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:28.085277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675ad74-9776-4d9b-bf35-18c196cf434a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:28.096577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd0c4e-2162-48ff-9a1e-a72c5838b0fb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:28.123666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:28.124251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.124542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.124823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.125661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] RBAC: Adding query filter params (role.id=c38c7f8d093b411e9b231ea8733760a4, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3a892d7213414aa589db028d1b7c27c2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:28.125975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:28.127403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:28.140193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 458/909] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/role_assignments?user.id=3a892d7213414aa589db028d1b7c27c2&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=c38c7f8d093b411e9b231ea8733760a4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:28.143049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:28.143747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.144029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.144336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.145064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:28.145370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:28.146772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:28.157726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 452/910] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:28.159787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:28.269214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cbee5-76df-4d29-900a-ad58de2bc9a6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:28.306349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:28.307018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.307302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.307581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.308423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] RBAC: Adding query filter params (interface=public, service_id=0ecebdfa7f954001975fde3dd39e9203, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:28.308736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:28.310686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccd9af-5f12-4abb-aaf3-2311cd6613c2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:28.322094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 459/911] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:28 2022] GET /identity/v3/endpoints?service_id=0ecebdfa7f954001975fde3dd39e9203&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:28.322650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:28.431749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c612a5-6cbc-4638-a25d-aac534655647 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:28.466239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0a36c-929f-4bfe-968d-08452e6f9080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:28.467279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a36c-929f-4bfe-968d-08452e6f9080 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.467761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a36c-929f-4bfe-968d-08452e6f9080 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.468269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0a36c-929f-4bfe-968d-08452e6f9080 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.469181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a36c-929f-4bfe-968d-08452e6f9080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:28.470723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0a36c-929f-4bfe-968d-08452e6f9080 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:28.477201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 453/912] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:28 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:32:28.584253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b042df-b3b6-4d17-a3be-d6ce0369f31b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.584570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b042df-b3b6-4d17-a3be-d6ce0369f31b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.584791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b042df-b3b6-4d17-a3be-d6ce0369f31b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.586532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 460/913] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:28.592709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.593046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.593266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.610673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:28.611871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:28.682693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 454/914] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:28.705879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b042df-b3b6-4d17-a3be-d6ce0369f31b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:28.807459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b042df-b3b6-4d17-a3be-d6ce0369f31b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:28.842432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:28.843150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:28.843484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:28.843759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:28.844490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:28.845897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:28.857982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:28.859026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:28.860591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 461/915] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:28.865564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:28.965869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca94afc-baa8-4510-80bb-e28f0b1bc764 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:29.001509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:29.002548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.003085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.003557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.004495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:29.005002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:29.006553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:29.011918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 455/916] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:29.018060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:29.122473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725456f9-1afe-4459-8f10-c88f4cf3822e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:29.155642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:29.156344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.156628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.156902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.157607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:29.157914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:29.159267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec961c-403b-4b28-87d6-514d28b10356 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:29.164252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 462/917] 10.222.0.73 () {56 vars in 1114 bytes} [Thu May 12 17:32:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:29.170603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:29.270153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c1d54-6bbb-473d-b213-b5234d224575 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:29.276027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2e6d2-98bb-41c8-9f28-55b777da9e71 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.276350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2e6d2-98bb-41c8-9f28-55b777da9e71 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.276627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2e6d2-98bb-41c8-9f28-55b777da9e71 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.278202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 463/918] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:29.283642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.283922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.284209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.303324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:29.304520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:29.313874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:29.315280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.316089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.316627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.317897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:29.320225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:29.346935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 456/919] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:29.390278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 464/920] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:29.415769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:29.525727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5a36-a1d7-47ec-9414-0af8a23e43cf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:29.560743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:29.561896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.562402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.562894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.563742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:29.565324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:29.577971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:29.579080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:29.580121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:29.582650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 457/921] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:29.588538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:29.694608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dff3f-158c-4344-9ed0-b422a5ab48b6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:29.729121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:29.729784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.730060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.730389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.731096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:29.731968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:29.733561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:29.738732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 465/922] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:29.744379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:29.849570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f7a4f-12c5-4f99-b959-14dce7a8ccf1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:29.884524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:29.885191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:29.885471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:29.885750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:29.886455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:29.886757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:29.888772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:29.894235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 458/923] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:29.900154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:30.003614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548b52a-92b8-4575-a1e9-1dbe3e9d3dc2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:30.037713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:30.038370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.038662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.038947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.039728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] RBAC: Adding query filter params (interface=public, service_id=f6884ba45c77489cb2560c093125a2d3, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:30.040034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:30.041526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278f4c5-b2d4-4efa-b859-b2e887c2cb5c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:30.047074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 466/924] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:29 2022] GET /identity/v3/endpoints?service_id=f6884ba45c77489cb2560c093125a2d3&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:30.052224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:30.151710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de1df0-701b-4422-a496-31650ede1805 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:30.185242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd97ff-b14f-438b-915e-a09c0abce4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:30.185921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd97ff-b14f-438b-915e-a09c0abce4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.186195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd97ff-b14f-438b-915e-a09c0abce4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.186466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd97ff-b14f-438b-915e-a09c0abce4e3 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.187144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd97ff-b14f-438b-915e-a09c0abce4e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:30.188522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd97ff-b14f-438b-915e-a09c0abce4e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:30.194881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 459/925] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:30 2022] GET /identity/v3/services => generated 1497 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:32:30.478150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583af91f-e737-4905-8f20-6684590dfd34 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.478666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583af91f-e737-4905-8f20-6684590dfd34 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.479054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583af91f-e737-4905-8f20-6684590dfd34 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.480793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 467/926] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:30.486879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.487278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.487675 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.505913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:30.507098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:30.575916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 460/927] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:30.598857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583af91f-e737-4905-8f20-6684590dfd34 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:30.704331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583af91f-e737-4905-8f20-6684590dfd34 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:30.741248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:30.741975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.742250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.742519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.753450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:30.761599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:30.762540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:30.763779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 468/928] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 12 17:32:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:30.768872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:30.874512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970418d-978e-46e5-b8de-8e49dc23ff3f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:30.908627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:30.909294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.909572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.909843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.910585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:30.910883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:30.912258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39eaf3-c07e-4608-918e-9f2bcd5060d7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:30.921681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 461/929] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:32:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:30.927303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:30.986185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c7fa-8ec1-4123-b2d5-ba61c5469f5a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.986530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c7fa-8ec1-4123-b2d5-ba61c5469f5a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.986727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c7fa-8ec1-4123-b2d5-ba61c5469f5a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:30.988272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 462/930] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:30.993649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:30.993857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:30.994211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.010546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:31.011725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:31.035332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77e826-2083-4c6d-aae3-2faf0e259314 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.084184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 463/931] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:31.084739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.084739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.084739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.084739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.094063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.095545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.102610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:31.112703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:31.113635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:31.114938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:31.114938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:31.114938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:31.114938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:31.114938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 469/932] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:31.120360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.209499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8440-a5d4-4158-87c6-63946379fefc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.225096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9f61d-2204-4fdd-87b4-136ce81cf1d1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.244952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.246499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.247711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.248448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.249495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.251652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.260050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.260820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.261170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.261512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.262319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:31.262727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.264198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:31.268432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:31.269365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 12 17:32:31.270209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:31.278847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 464/933] 10.222.0.73 () {56 vars in 1132 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:31.281088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.281965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 470/934] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:31.287486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.384021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12e3cc-6cfb-4582-8180-16f2e2235a89 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.418405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b5a0d-c3c6-4363-9ab7-48b128e33360 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.418635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.419841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.420128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.420431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.421131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:31.421429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.422782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.432122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 465/935] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:31.433037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.460609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.462137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.462750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.463540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.474332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.476805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:31.498859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:31.499814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:31.500821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:31.506429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 471/936] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:31.507080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.542810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874576-a4d8-40e3-aa2f-46a67335e395 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.583854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.584578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.584860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.585134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.586127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:31.586881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.588762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.596529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 466/937] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:31.602499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.641384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499874-098f-4cce-9bef-3434d764a5ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.675061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.675784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.676066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.676377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.677265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:31.677576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.678988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.697201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 472/938] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:31.702932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.718868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357db6b-7739-4ed5-a0f6-03837583d662 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.764425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58918e03-c03b-4394-a6e2-0cace6755317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.765216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58918e03-c03b-4394-a6e2-0cace6755317 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.765506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58918e03-c03b-4394-a6e2-0cace6755317 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.765782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58918e03-c03b-4394-a6e2-0cace6755317 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.766913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58918e03-c03b-4394-a6e2-0cace6755317 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.768605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58918e03-c03b-4394-a6e2-0cace6755317 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.796367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 467/939] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:31.797968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4e99-7f77-49dc-9919-f578b8d5b837 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.798261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4e99-7f77-49dc-9919-f578b8d5b837 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.798594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4e99-7f77-49dc-9919-f578b8d5b837 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.800141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 468/940] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:31.805995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.806302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.806557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.814969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5ce8-bf46-4282-a593-fe932293b86a None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:31.825313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:31.826643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:31.868893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:31.869599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:31.869884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:31.870159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:31.882633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:31.884182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:31.907325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:31.908621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:31.909547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:31.909547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:31.909547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:31.909547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:31.912787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 473/941] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:31.914857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:31.923686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 469/942] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:31.948800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.046483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9655b-8f8f-45a0-97d1-5a6cafdb49a6 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.080820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bc97e-debc-4ff8-b792-d81fdc9fb8b9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.093028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.093693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.093972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.094249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.094970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:32.095629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.097340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.112109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 474/943] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:32.117779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.126794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.128106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.129072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.129570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.130894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.133164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.135397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING keystone.server.flask.common [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 12 17:32:32.159014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 12 17:32:32.159014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 12 17:32:32.159014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [parameters: {'id': 'dd3761f276ff4c28b46c9240cef51786', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 12 17:32:32.159014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd3761f276ff4c28b46c9240cef51786.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd3761f276ff4c28b46c9240cef51786. May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 17:32:32.181002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 12 17:32:32.181877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return project_ref.to_dict() May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() May 12 17:32:32.182837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 12 17:32:32.183793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application c = connection._execute_20( May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 12 17:32:32.185632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 17:32:32.186525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'id': 'dd3761f276ff4c28b46c9240cef51786', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.187438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:32.194739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict( May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd3761f276ff4c28b46c9240cef51786. May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 470/944] 10.222.0.73 () {60 vars in 1140 bytes} [Thu May 12 17:32:31 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:32.197194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.247656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a810893-ffe8-4f6c-92b6-4831102f4fd1 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.288828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.290537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.291006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.291483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.304906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.306725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.317480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f46d8a-8149-485a-a6a0-00968d92847a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:32.329135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:32.330116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:32.331007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:32.336003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 475/945] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:32.337556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.368004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.369671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.370437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.370917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.380033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.382766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:32.398534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:32.399473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:32.400753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 471/946] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:32.406134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.448435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98103a71-19be-4f32-8d7c-11865a6b0bf2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.483657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.484395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.484627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.485004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.485796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:32.486099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.487562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1af017-2993-4562-b5ac-059c19cb0a8e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.500203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 476/947] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:32.501604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0911cc-8250-4916-ba80-8078256823ec None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.501895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0911cc-8250-4916-ba80-8078256823ec None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.502173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0911cc-8250-4916-ba80-8078256823ec None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.503722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 477/948] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:32.505925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0911cc-8250-4916-ba80-8078256823ec None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.529710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2049929-41a8-4537-9a56-8d1b5d80ecbb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.564333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.564908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.565189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.565463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.566320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:32.566635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.568039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c74a61-0779-4b9c-ab0f-f29c0271c4b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.588699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 472/949] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:32.593924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.594242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.594479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.627306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0911cc-8250-4916-ba80-8078256823ec None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.632155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:32.633787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:32.675977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.677293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.677778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.678228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/3a892d7213414aa589db028d1b7c27c2/roles/c38c7f8d093b411e9b231ea8733760a4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.686152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=3a892d7213414aa589db028d1b7c27c2, role_id=c38c7f8d093b411e9b231ea8733760a4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.688453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.708081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 478/950] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:32:32 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/3a892d7213414aa589db028d1b7c27c2/roles/c38c7f8d093b411e9b231ea8733760a4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:32:32.723173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 473/951] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:32.749218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:32.867150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73bf161-a7e5-4a55-80da-4f7a2f22b623 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:32.906965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:32.907785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:32.908064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:32.908377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:32.909036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:32.910393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:32.924008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:32.925082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:32.925954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 479/952] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:32.933111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:33.032883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338d47-a33e-4523-b43d-c5808e0a5a9b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:33.068775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:33.069443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:33.069722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:33.069989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:33.070691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:33.071046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:33.072371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:33.078323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 474/953] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:33.084013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:33.185956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b994307-896f-4a31-b41a-ac208fcbb170 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:33.220574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:33.221240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:33.221524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:33.221801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:33.222510 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:33.222814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:33.224190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce21-df2a-46d2-823c-36a6079ace60 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:33.229445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 480/954] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:33.235570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:33.339749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a43c6-5d6b-447f-a298-92ebc0304391 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:33.373804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae15fcd-dadd-48eb-acf4-88674abf5bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:33.374466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae15fcd-dadd-48eb-acf4-88674abf5bc6 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:33.374746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae15fcd-dadd-48eb-acf4-88674abf5bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:33.375019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae15fcd-dadd-48eb-acf4-88674abf5bc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:33.375759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae15fcd-dadd-48eb-acf4-88674abf5bc6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:33.377128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae15fcd-dadd-48eb-acf4-88674abf5bc6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:33.399564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 475/955] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:34.248200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e7f42-a477-439d-b9ec-faeff875453a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:34.248753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e7f42-a477-439d-b9ec-faeff875453a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:34.248753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e7f42-a477-439d-b9ec-faeff875453a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:34.250231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 481/956] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:34.255748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:34.256146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:34.256388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:34.272909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:34.274092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:34.349670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 476/957] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:34.376554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e7f42-a477-439d-b9ec-faeff875453a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:34.483804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e7f42-a477-439d-b9ec-faeff875453a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:34.520252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:34.521282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:34.521730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:34.522151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:34.522964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:34.524489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:34.536904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:34.538120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:34.539167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:34.546933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 482/958] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:34.549268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:34.665212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bee10d-6f5d-4f4c-b058-022c583d7b67 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:34.701678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:34.702342 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:34.702622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:34.702895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:34.703595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:34.703898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:34.705287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746217-2607-41b8-bd72-e0c981451904 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:34.710394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 477/959] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:34.716040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:34.816012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72725a1-0199-4262-a3d7-f5163495dc1d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:34.853995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:34.855020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:34.855519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:34.855988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:34.857496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:34.858079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:34.859629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0510-3280-4a37-9de9-752295c0e0e4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:34.872831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 483/960] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:35.021039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e4b70-7a21-4292-b4ee-ebc3ca5eb108 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.021406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e4b70-7a21-4292-b4ee-ebc3ca5eb108 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.021599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e4b70-7a21-4292-b4ee-ebc3ca5eb108 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.023225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 478/961] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:35.029620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.029916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.030166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.047626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:35.048668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:35.112229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccb3cd-5c6f-4481-9c4e-9ab9a03b7770 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.112530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccb3cd-5c6f-4481-9c4e-9ab9a03b7770 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.112717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccb3cd-5c6f-4481-9c4e-9ab9a03b7770 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.114302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 479/962] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:35.115729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 484/963] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:35.119668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.119951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.120249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.139036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:35.140243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:35.143316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.207311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 480/964] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:35.231085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.239864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f830-1b21-4f07-a6b2-f3991d5d8ba9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.274264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.275036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.275205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.275469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.276198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:35.277619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:35.279474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING keystone.server.flask.common [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 12 17:32:35.293147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.sql.core [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 12 17:32:35.293147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 12 17:32:35.293147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [parameters: {'user_id': 'f71cfe5c83874051baa61c8163ffaa18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 12 17:32:35.293147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80296) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() May 12 17:32:35.303876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 17:32:35.305749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application yield resource May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application next(self.gen) May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.rollback() May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.force_reraise() May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise self.value May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application session.commit() May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 12 17:32:35.307106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._prepare_impl() May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.session.flush() May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._flush(objects) May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application compat.raise_( May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application flush_context.execute() May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rec.execute(self) May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 12 17:32:35.308153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application _emit_insert_statements( May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = connection._execute_20( May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ret = self._execute_context( May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 12 17:32:35.309305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self.dialect.do_execute( May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result = self._query(query) May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application conn.query(q) May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application result.read() May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 12 17:32:35.310528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application packet.raise_for_error() May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f71cfe5c83874051baa61c8163ffaa18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:35.311576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 12 17:32:35.317868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 485/965] 10.222.0.73 () {60 vars in 1131 bytes} [Thu May 12 17:32:35 2022] POST /identity/v3/users => generated 160 bytes in 163 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:35.320022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.341244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324786a-035e-49a2-bbfa-02e9fb017abc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.377967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.379175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.379711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.380206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.391016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:35.399718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:35.400747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:35.401595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:35.405098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 481/966] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:35.410771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.425410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d1823-7a23-43a6-9ecd-14d89845dd10 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.459621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37498f-d16c-43fe-a371-312378162761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.460391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37498f-d16c-43fe-a371-312378162761 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.460676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37498f-d16c-43fe-a371-312378162761 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.460952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37498f-d16c-43fe-a371-312378162761 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.468991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37498f-d16c-43fe-a371-312378162761 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:35.470629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37498f-d16c-43fe-a371-312378162761 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-af37498f-d16c-43fe-a371-312378162761 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:35.486612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:35.487604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 12 17:32:35.488633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:35.492482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 486/967] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:35.493428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37498f-d16c-43fe-a371-312378162761 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.521410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1921af-4ef2-4143-9919-89b26ca57ce5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.560895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.562056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.562548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.563037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.564017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:35.564569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:35.566406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:35.580341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 482/968] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:35.582777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.598888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37498f-d16c-43fe-a371-312378162761 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.634559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.635157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.635438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.635712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.636607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:35.636853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:35.638857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93808a1-16eb-4afe-8212-2474e5174d7b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:35.685226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 487/969] 10.222.0.73 () {56 vars in 1101 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:35.696760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fb666-f4b4-473d-9f88-34dd665a741b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.735227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.736538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.737075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.737577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.752676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6bcaa-4768-49a5-9651-0f2aaec13ee7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.753004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6bcaa-4768-49a5-9651-0f2aaec13ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.753236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6bcaa-4768-49a5-9651-0f2aaec13ee7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.753515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:35.754873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 488/970] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:35.755402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:35.761938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.762554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.763149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:35.774227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:35.775304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:35.776373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:35.776373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:35.776373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:35.776373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:35.776373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 483/971] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:35.782215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.785876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:35.787804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:35.869050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 489/972] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:35.887036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:35.890155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29da1d6-3fb6-440d-a9fb-713282d5fd87 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:35.932493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:35.933545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:35.933891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:35.934229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:35.935713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:35.936086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:35.937544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:35.952583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 484/973] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:35.958027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.000960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5ed0-b1d3-4573-a103-dc1e0193501b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.034943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.036094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.036476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.036823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.037553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.038986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:36.051601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:36.053018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:36.054149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:36.055469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 490/974] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:36.061505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.067186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1fe3-7bdb-4cda-8215-d0e55b5e7afe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.100019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.100852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.101133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.101401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.108800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.110224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:36.125691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:36.126659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:36.127612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 485/975] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:32:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:36.133072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.163123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603435d-0c3c-4ec8-bb39-83de319f484c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.197660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.198322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.198605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.198884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.199586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:36.199896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.201394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.206255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 491/976] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:36.211600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.233064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18d0a-388b-4e46-b4b1-5a9cf0899d6a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.266764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.267430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.267712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.267993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.268849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:36.269155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.270571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.291943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 486/977] 10.222.0.73 () {56 vars in 1149 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:36.292687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.309219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51960-47c5-479f-a253-864fdf0221d3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.341392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.342053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.342336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.342611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.343310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:36.343617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.344997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.349559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 492/978] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:36.355211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.394436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052055e-266a-4043-80c5-04b617451a00 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.427415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.427991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.428295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.428573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.451053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197afdb-ebfd-4d7b-9662-d36c7cf56767 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.484339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.484995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.485276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.485548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.486340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] RBAC: Adding query filter params (interface=admin, service_id=f6884ba45c77489cb2560c093125a2d3, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:36.486657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.487998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.493317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 493/979] 10.222.0.73 () {56 vars in 1249 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/endpoints?service_id=f6884ba45c77489cb2560c093125a2d3&interface=admin®ion_id=RegionOne => generated 193 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:36.498536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.500125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.501590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.594410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de849-f09b-454f-ac9a-c2019c57dd15 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.605317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 487/980] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:36.611506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.635859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.636559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.636842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.637119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.638413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.639769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.645611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 494/981] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:32:36.718193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cf6b4-3f41-41de-93d2-53744ca9733f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.755911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.756749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.757028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.757294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.769622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.771086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:36.789877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:36.790868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:36.797025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:36.797025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:36.797025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:36.797025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:36.797025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 488/982] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:36.797309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:36.900289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998ddd2-bf9c-4511-b5b8-f0327c7c8838 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:36.932639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:36.933309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:36.933596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:36.933874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:36.935651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:36.935979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:36.937378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:36.953803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 495/983] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:36.955067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:37.067297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fed609-3c40-4a4b-971c-58c58eed95ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:37.102865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:37.103657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:37.103936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:37.104234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:37.111171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:37.112637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0af1064e-6465-4c2a-bb01-62a8aa9c0aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:37.126851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:37.127777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:37.128813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 489/984] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:37.135246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:37.159911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4212dc-a7cb-4c5d-aff7-f3e9ca3d4586 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:37.160270 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4212dc-a7cb-4c5d-aff7-f3e9ca3d4586 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:37.160506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4212dc-a7cb-4c5d-aff7-f3e9ca3d4586 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:37.161984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 490/985] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:37.166764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:37.167049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:37.167323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:37.183433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:37.184770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:37.236826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f2a34-3b8e-469e-9815-b205afa59f54 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:37.249934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 491/986] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:37.270957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:37.271622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:37.271956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:37.272055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:37.272278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:37.273056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:37.273419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:37.274886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:37.291664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 496/987] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:37.292924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:37.369599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e687aa-3685-44c0-a84f-94ce180eb151 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:37.393616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd75267-9ce8-4729-afae-cb45fb6abfe8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:37.412337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c09983-74c1-45e7-8a02-e7376c7885a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:37.413051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c09983-74c1-45e7-8a02-e7376c7885a9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:37.413331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c09983-74c1-45e7-8a02-e7376c7885a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:37.413598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c09983-74c1-45e7-8a02-e7376c7885a9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:37.414400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c09983-74c1-45e7-8a02-e7376c7885a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:37.415755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c09983-74c1-45e7-8a02-e7376c7885a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:37.431356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:37.432018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:37.432333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:37.432606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:37.433451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] RBAC: Adding query filter params (role.id=c38c7f8d093b411e9b231ea8733760a4, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3a892d7213414aa589db028d1b7c27c2) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:37.433736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:37.435318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e88dc7-1848-4424-8e91-4e03071ee704 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:37.437858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 492/988] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:37 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:37.446421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 497/989] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:37 2022] GET /identity/v3/role_assignments?user.id=3a892d7213414aa589db028d1b7c27c2&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=c38c7f8d093b411e9b231ea8733760a4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:38.924281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b6f9-e405-4fe4-897d-e7eb79987f4f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:38.924857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b6f9-e405-4fe4-897d-e7eb79987f4f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:38.924857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231b6f9-e405-4fe4-897d-e7eb79987f4f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:38.926186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 493/990] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:38.933862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:38.935042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:38.935042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:38.953066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:38.955179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:39.028211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 498/991] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:39.047801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231b6f9-e405-4fe4-897d-e7eb79987f4f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:39.144936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231b6f9-e405-4fe4-897d-e7eb79987f4f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:39.177021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:39.177739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.178015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.178286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.178926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:39.180302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:39.192702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:39.193686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:39.199907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 494/992] 10.222.0.73 () {56 vars in 1126 bytes} [Thu May 12 17:32:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:39.201387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:39.295991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32983bf4-86dc-41a8-903b-afa9be42bef4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:39.328150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:39.328847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.329219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.329399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.330093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:39.330394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:39.331742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:39.341483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 499/993] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:39.342674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:39.437831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966825a0-f468-473d-886a-2dab2496cce1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:39.469476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:39.470147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.470423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.470683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.471384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:39.471682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:39.473053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36734d-6634-4f23-9053-79c935b04a67 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:39.482573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 495/994] 10.222.0.73 () {56 vars in 1122 bytes} [Thu May 12 17:32:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:39.483622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:39.578175 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44cf12-63fb-43fa-b041-14d764e67b56 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:39.613322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eabacf-09e5-4fdb-bc6d-e8110af1108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:39.614337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eabacf-09e5-4fdb-bc6d-e8110af1108d None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.614847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eabacf-09e5-4fdb-bc6d-e8110af1108d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.615324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eabacf-09e5-4fdb-bc6d-e8110af1108d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.616578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eabacf-09e5-4fdb-bc6d-e8110af1108d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:39.617983 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eabacf-09e5-4fdb-bc6d-e8110af1108d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:39.636541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 500/995] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:32:39.768173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6456881-cd9c-4e29-ae2f-6764ce5065af None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.768484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6456881-cd9c-4e29-ae2f-6764ce5065af None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.768672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6456881-cd9c-4e29-ae2f-6764ce5065af None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.770185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 496/996] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:39.775925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.776233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.776513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.793575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:39.794822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:39.845089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4bc8d-8cc7-46c2-9a38-6591d8c0bc93 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.845494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4bc8d-8cc7-46c2-9a38-6591d8c0bc93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.845844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4bc8d-8cc7-46c2-9a38-6591d8c0bc93 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.847592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 497/997] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:39.852389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:39.852668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:39.852946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:39.869957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 501/998] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:39.871396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:39.873185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:39.893068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:39.947054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 498/999] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:39.969966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:39.992559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7797195-ec42-4995-99c5-09b0ea4d1644 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.031056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.031757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.032033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.032335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] PATH_INFO: `/v3/services/image` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.032980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.034329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:40.045528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:40.046765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:40.047821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:40.052200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 502/1000] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 12 17:32:39 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:40.052690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:40.073410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71cc24-04a2-4cfb-a859-b4e4a1bcfe73 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.107947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.108773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.109055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.109330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.109977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.111336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:40.123543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:40.124609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:40.125435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 499/1001] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:40.130753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:40.151164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f7e95-4f86-4a82-8bb4-6014e85bad42 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.185529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.186365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.186791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.187164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.187975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] RBAC: Adding query filter params (name=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:40.188308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.189662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.194767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 503/1002] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:40.200770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:40.230079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ccad7-0602-404e-9e86-2d24104a0bc4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.263375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.264202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.264576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.264941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.265733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:40.266134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.267570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.272609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 500/1003] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:40.278024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:40.304093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56748-8790-4862-b210-c8c16201fb44 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.336935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.337597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.337888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.338159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.338864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] RBAC: Adding query filter params (type=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:40.339184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.340557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.345634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 504/1004] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:40.384308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58ad87-db2a-4f1c-9a79-06b31f678faa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.421020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.421700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.421984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.422260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.422976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:40.423284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.424708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.430548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 501/1005] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:40.437357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:40.540730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d0d5d-f980-434a-8831-fafbab624e73 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.574099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.574673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.574976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.575912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.576753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] RBAC: Adding query filter params (interface=public, service_id=1ce7e8ebfda149c18e52ad6c281a11c0, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:40.577068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.578439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce82b8b-aebe-4fec-a615-cc3c3e2a3052 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.584677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 505/1006] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:40 2022] GET /identity/v3/endpoints?service_id=1ce7e8ebfda149c18e52ad6c281a11c0&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:40.589952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:40.685815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0240323-aacf-4a29-aad5-5507626b9ba5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:40.717323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cad7ef-5685-462e-9e7d-a093b3f46226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:40.717981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cad7ef-5685-462e-9e7d-a093b3f46226 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:40.718258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cad7ef-5685-462e-9e7d-a093b3f46226 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:40.718529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cad7ef-5685-462e-9e7d-a093b3f46226 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:40.719198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cad7ef-5685-462e-9e7d-a093b3f46226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:40.720570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cad7ef-5685-462e-9e7d-a093b3f46226 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:40.732192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 502/1007] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:40 2022] GET /identity/v3/services => generated 1727 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:32:41.927807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2269d0-ed41-421d-ae65-25dc0320f819 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:41.928365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2269d0-ed41-421d-ae65-25dc0320f819 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:41.928442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2269d0-ed41-421d-ae65-25dc0320f819 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:41.929891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 506/1008] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:41.935438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:41.935866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:41.936255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:41.958386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:41.959567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:42.032216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 503/1009] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:42.055038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2269d0-ed41-421d-ae65-25dc0320f819 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:42.150602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2269d0-ed41-421d-ae65-25dc0320f819 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:42.183018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef263eb-8127-4fd3-b60c-0a39d307ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:42.183724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef263eb-8127-4fd3-b60c-0a39d307ec99 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.184008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef263eb-8127-4fd3-b60c-0a39d307ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.184327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef263eb-8127-4fd3-b60c-0a39d307ec99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.185599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef263eb-8127-4fd3-b60c-0a39d307ec99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:42.187001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef263eb-8127-4fd3-b60c-0a39d307ec99 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:42.188798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 507/1010] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:42.194405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:42.287026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62473755-04f8-4502-ad23-f6caa9091d89 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:42.318087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0422b-4ae2-42fd-a1d7-cff93c4c4626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:42.318737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0422b-4ae2-42fd-a1d7-cff93c4c4626 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.319010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0422b-4ae2-42fd-a1d7-cff93c4c4626 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.319273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0422b-4ae2-42fd-a1d7-cff93c4c4626 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.319976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0422b-4ae2-42fd-a1d7-cff93c4c4626 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:42.321406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0422b-4ae2-42fd-a1d7-cff93c4c4626 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:42.352197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 504/1011] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:42 2022] POST /identity/v3/projects => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:32:42.613958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469fc07-a6bb-4345-a829-ba58b0f3b95a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.614144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469fc07-a6bb-4345-a829-ba58b0f3b95a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.614363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469fc07-a6bb-4345-a829-ba58b0f3b95a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.616234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 508/1012] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:42.621618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f7ea6-2f05-4380-9217-caf4a0f93c7e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.622469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f7ea6-2f05-4380-9217-caf4a0f93c7e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.622900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f7ea6-2f05-4380-9217-caf4a0f93c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.639722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e39f7ea6-2f05-4380-9217-caf4a0f93c7e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:42.640925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e39f7ea6-2f05-4380-9217-caf4a0f93c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:42.706107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 505/1013] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:42 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:42.728791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469fc07-a6bb-4345-a829-ba58b0f3b95a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:42.837349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469fc07-a6bb-4345-a829-ba58b0f3b95a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:42.872119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd088e58-8a13-4273-90d1-1aa519dd8def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:42.873077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088e58-8a13-4273-90d1-1aa519dd8def None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.873535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088e58-8a13-4273-90d1-1aa519dd8def None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.873953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088e58-8a13-4273-90d1-1aa519dd8def None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.874823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd088e58-8a13-4273-90d1-1aa519dd8def None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:42.876401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd088e58-8a13-4273-90d1-1aa519dd8def None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:42.900192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 509/1014] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:32:42 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:42.970297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336410c-485d-4501-a4a8-7e7ba69be279 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.970840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336410c-485d-4501-a4a8-7e7ba69be279 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.970840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336410c-485d-4501-a4a8-7e7ba69be279 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.972405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 506/1015] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:42.977949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:42.978436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:42.978545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:42.996029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:42.997517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:43.071366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 510/1016] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:42 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:43.096722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336410c-485d-4501-a4a8-7e7ba69be279 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:43.198126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336410c-485d-4501-a4a8-7e7ba69be279 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:43.231479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:43.232382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:43.232668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:43.232941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:43.233588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:43.234947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:43.246712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:43.247785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 12 17:32:43.248665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:43.250027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 507/1017] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:43.254581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:43.348894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd981-e909-4131-a2e8-ed640939dbde None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:43.381078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:43.381742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:43.382024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:43.382299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:43.383001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:43.383305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:43.384686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:43.394561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 511/1018] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:43.396259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:43.492016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bb157-d4f0-48bc-a9bc-7fb5f68b6091 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:43.525047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:43.525720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:43.526002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:43.526277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:43.526986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:43.527292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:43.529258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bfa26-b207-4b15-be37-e20a7d2f2174 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:43.538903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 508/1019] 10.222.0.73 () {56 vars in 1112 bytes} [Thu May 12 17:32:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:43.540365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:43.638726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9c9dd-c47e-4a2b-86ec-81199536bd92 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:43.672729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3ba7f-f239-4ca2-8cf2-de1136f1ada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:43.673387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3ba7f-f239-4ca2-8cf2-de1136f1ada0 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:43.673670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3ba7f-f239-4ca2-8cf2-de1136f1ada0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:43.673945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3ba7f-f239-4ca2-8cf2-de1136f1ada0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:43.674673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3ba7f-f239-4ca2-8cf2-de1136f1ada0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:43.676023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3ba7f-f239-4ca2-8cf2-de1136f1ada0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:43.704196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 512/1020] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:32:44.650073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025ca13-cfa8-4e83-a3c5-e67513de8374 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:44.650589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025ca13-cfa8-4e83-a3c5-e67513de8374 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:44.650701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025ca13-cfa8-4e83-a3c5-e67513de8374 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:44.652127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 509/1021] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:44.657926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:44.658338 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:44.658500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:44.675089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:44.676323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:44.747181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 513/1022] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:44.771901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025ca13-cfa8-4e83-a3c5-e67513de8374 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:44.872607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025ca13-cfa8-4e83-a3c5-e67513de8374 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:44.905787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de259695-d72c-4e7a-aead-a3b0b50aae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:44.906611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de259695-d72c-4e7a-aead-a3b0b50aae36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:44.906894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de259695-d72c-4e7a-aead-a3b0b50aae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:44.907165 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de259695-d72c-4e7a-aead-a3b0b50aae36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:44.908472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de259695-d72c-4e7a-aead-a3b0b50aae36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:44.909942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de259695-d72c-4e7a-aead-a3b0b50aae36 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:44.911843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 510/1023] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:44.917675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:45.017445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83821db-c160-4688-bca8-ec1fd166847e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:45.050957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9265374-ac1f-4ca7-8b9e-e728acf94699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:45.051630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9265374-ac1f-4ca7-8b9e-e728acf94699 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.051915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9265374-ac1f-4ca7-8b9e-e728acf94699 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.052211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9265374-ac1f-4ca7-8b9e-e728acf94699 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.052915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9265374-ac1f-4ca7-8b9e-e728acf94699 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:45.054339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9265374-ac1f-4ca7-8b9e-e728acf94699 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:45.076449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 514/1024] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:32:44 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:32:45.182067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8f49-6dd3-4aa5-8c7d-d6415a9f42d2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.182402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8f49-6dd3-4aa5-8c7d-d6415a9f42d2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.182634 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8f49-6dd3-4aa5-8c7d-d6415a9f42d2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.184177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 511/1025] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:45.190204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.190511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.190767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.208668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:45.209891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:45.278092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 515/1026] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:45.301118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde8f49-6dd3-4aa5-8c7d-d6415a9f42d2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:45.400960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde8f49-6dd3-4aa5-8c7d-d6415a9f42d2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:45.433023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:45.433780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.434057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.435016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.435818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:45.437273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:45.448959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:45.449972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:45.451192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 512/1027] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 12 17:32:45 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:45.458210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:45.552558 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f3133-d6ff-4ee0-b94e-111b7d21353a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:45.589847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:45.590542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.591304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.591572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.592310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] RBAC: Adding query filter params (name=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:45.592613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:45.594135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:45.603595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 516/1028] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:45.605012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:45.704315 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e611e6-6e59-41df-b1d1-3f06bd94b81d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:45.746159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:45.747689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.748185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.748648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.749566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:45.750094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:45.751629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:45.756844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 513/1029] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:45.762927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:45.864304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857f48e-a1ed-4418-aebf-12b26d136723 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:45.897826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:45.898778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:45.899233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:45.899678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:45.900697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] RBAC: Adding query filter params (interface=public, service_id=d9c2b1b06bdf47d6829281c19f582a67, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:45.901186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:45.902697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8967fc-a355-433d-8620-694b367938d5 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:45.908940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 517/1030] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:32:45 2022] GET /identity/v3/endpoints?service_id=d9c2b1b06bdf47d6829281c19f582a67&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:45.914049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:46.009531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b8b5-3b24-4da3-8649-1cda0c1dd2ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:46.035582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7af787-4565-489b-809b-00eec880bd4a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.036078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7af787-4565-489b-809b-00eec880bd4a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.036642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7af787-4565-489b-809b-00eec880bd4a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.039581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 518/1031] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:46.044571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:46.044727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.044916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.045187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.045284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.045497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.045765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.046488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:46.047777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:46.054431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 514/1032] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:32:45 2022] GET /identity/v3/services => generated 1957 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:32:46.061642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:46.062856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:46.133085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 519/1033] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:46.157677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:46.259894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a6e7-083d-41d6-ad12-01883896afd6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:46.292371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:46.293204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.293483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.294306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.305496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:46.313619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:46.314549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:46.315401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 515/1034] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 12 17:32:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:46.321084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:46.415110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5381ca-fef4-4ed5-ad1f-5e8be1576ed1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:46.447417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:46.448085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.448384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.448659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.449400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:46.449706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:46.451239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:46.459249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 520/1035] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:32:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:46.464673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:46.559793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48025ea-613f-4034-a1fc-de2e0afcd241 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:46.598066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:46.598906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.599185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.599459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.610616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:46.614107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:46.631251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:46.632206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:46.633124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:46.633124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:46.633124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:46.633124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:46.633436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 516/1036] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:46.639137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:46.732274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c892f8-dcc1-4937-a8ad-16fc599c4559 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:46.764073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:46.764856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.765043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.765308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.766021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:46.766323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:46.767661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:46.779225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 521/1037] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:46.784577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:46.881830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3d8aa-33a3-42b2-8d1d-4d49d7fb7b77 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:46.913348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:46.914130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:46.914406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:46.914673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:46.922651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:46.924073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:46.938152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:46.939115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:46.940068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:46.941343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 517/1038] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:32:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:46.945039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.039789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf1c8d-37e7-4397-9a8b-29f4ac8664cf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.071925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.072609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.072888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.073159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.073985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:47.074288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:47.076323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:47.097876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 522/1039] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:32:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:47.099214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.196642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950b6e8-88bc-4676-8422-b44733be721e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.229265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1c684-40be-44ef-9143-898d09e00a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.229933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c684-40be-44ef-9143-898d09e00a66 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.230213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c684-40be-44ef-9143-898d09e00a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.230483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c684-40be-44ef-9143-898d09e00a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.302467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1c684-40be-44ef-9143-898d09e00a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:47.303908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1c684-40be-44ef-9143-898d09e00a66 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:47.404268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 518/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 12 17:32:47.405665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.425091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a69c-600e-43a7-b2f7-7c50a1e6366f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.425400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a69c-600e-43a7-b2f7-7c50a1e6366f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.425652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878a69c-600e-43a7-b2f7-7c50a1e6366f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.427139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 519/1041] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:47.431995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.432315 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.432595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.448518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:47.449621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:47.504042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb6719-83ee-46d6-a3bd-0c2db99ae7b9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.518249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 520/1042] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:47.536861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.537642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.537919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.538187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.541808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.548006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:47.549532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:47.566545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:47.567418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:47.568512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:47.568512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:47.568512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:47.568512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:47.568512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 523/1043] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:47.573123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.645683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276867-b452-4808-b871-e9f6fbeb6d1c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.678522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1eb01-b6f2-41fa-ae2d-3a9e80d01de8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.689395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.690242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.690526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.690859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.704094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:47.711890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:47.713061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:47.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 521/1044] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:47.717698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.718357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.718639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.718913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.719639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:47.719944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:47.720623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.721333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:47.733716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 524/1045] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:47.739101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.826212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1e789-0fb7-41f8-822d-915cbc948a1c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.839655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77cf05-7552-406f-8e03-909b890aa295 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:47.859433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.860098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.860404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.860677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.861416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:47.861716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:47.863056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:47.871957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 522/1046] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:47.874708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:47.875495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:47.875770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:47.876037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:47.877697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.883209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:47.884687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:47.899178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:47.905942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 525/1047] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:47.906888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:47.976759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a48167-56a2-468f-a892-6fba47e3ee75 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.004316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e4dfa-ba99-4935-ab72-ee839a4498dc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.011154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.012029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.013303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.013588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] PATH_INFO: `/v3/users/4d6c97d8b9d0411d80b7ada4244c3e77` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.033707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] RBAC: Authorizing `identity:get_user(user_id=4d6c97d8b9d0411d80b7ada4244c3e77)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.035153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.041629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 523/1048] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/users/4d6c97d8b9d0411d80b7ada4244c3e77 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:48.041777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.041777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.041777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.041777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.042066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:48.042066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.042473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.042859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.055710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 526/1049] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:48.062374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.141763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07fa5-3e47-485f-9f56-5b008a760204 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.168728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8b92-b6a1-4f26-9dd7-1523c0aa312d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.174958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.175598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.175870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.176138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.201855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.202508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.202787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.203057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.203876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] RBAC: Adding query filter params (role.id=c38c7f8d093b411e9b231ea8733760a4, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=62c3b98740d14ab49d91783086150e88) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:48.204207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.205633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3379f2b-7205-4e6a-be3f-37dcfdf17c86 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.214146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 527/1050] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/role_assignments?user.id=62c3b98740d14ab49d91783086150e88&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=c38c7f8d093b411e9b231ea8733760a4 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:48.242957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.244354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.307907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf87eb9-8ee6-489d-9462-f048ba92f38a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.308205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf87eb9-8ee6-489d-9462-f048ba92f38a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.308492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf87eb9-8ee6-489d-9462-f048ba92f38a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.310985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 528/1051] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:48.316451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.316813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.317096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.335036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:48.336288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:48.352359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 524/1052] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:48.354135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.407438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 529/1053] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:48.430185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.454278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e4a5-c18f-41be-88b7-85ec2f6067ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.487268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.488122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.488439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.488710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.489874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] RBAC: Authorizing `identity:get_project(project_id=6bbe7ea67e964b29ad51f1fb70c15f65)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.491546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.493663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 525/1054] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:48.499364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.527398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b07a8-5dec-4f61-aa24-a577b6a5dd97 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.561999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.562783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.563063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.563334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] PATH_INFO: `/v3/services/image` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.563979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.565358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:48.577456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:48.578486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 12 17:32:48.579361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:48.584468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 530/1055] 10.222.0.73 () {56 vars in 1105 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:48.584719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.596278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5de35-53fb-4156-a8f3-1e291aff044d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.629739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.630393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.630671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.630944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.631762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65, user.id=4d6c97d8b9d0411d80b7ada4244c3e77) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:48.632073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.633517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.639356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 526/1056] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/role_assignments?user.id=4d6c97d8b9d0411d80b7ada4244c3e77&scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:48.689894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b2b25-4c70-4380-b12d-ce6b20b1c475 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.726578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.727500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.727926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.728388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.729270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] RBAC: Adding query filter params (name=image) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:48.729716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.731334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.741853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 531/1057] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:48.742970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.848367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccab73-390d-4bc0-965c-c3035fd933ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:48.879960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:48.880647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:48.880926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:48.881201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:48.881902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:48.882204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:48.883556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774c4d3-bc59-48c3-8674-36ab55d3edfa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:48.888857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 527/1058] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:48.894735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:48.990377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedccd1-de3f-42b8-9c68-f59d9e5afc7f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:49.023044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6d12-5a2d-4311-b2d8-69209157eab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:49.023702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6d12-5a2d-4311-b2d8-69209157eab9 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:49.023983 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6d12-5a2d-4311-b2d8-69209157eab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:49.024279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6d12-5a2d-4311-b2d8-69209157eab9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:49.025011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6d12-5a2d-4311-b2d8-69209157eab9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:49.026373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6d12-5a2d-4311-b2d8-69209157eab9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:49.047506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 532/1059] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:32:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:32:50.468428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d5c19-870e-474e-ab65-7fc8bec2d241 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:50.468985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d5c19-870e-474e-ab65-7fc8bec2d241 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:50.468985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d5c19-870e-474e-ab65-7fc8bec2d241 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:50.470476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 528/1060] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:50.475669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:50.476072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:50.476282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:50.492760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:50.493942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:50.563247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 533/1061] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:50.587190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d5c19-870e-474e-ab65-7fc8bec2d241 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:50.687065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d5c19-870e-474e-ab65-7fc8bec2d241 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:50.719976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:50.721086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:50.721510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:50.721946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:50.734745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:50.743512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:50.744605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:50.745449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:50.748252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 529/1062] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 12 17:32:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:50.752381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:50.849531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cc8f1-2128-4ed0-aebb-af50cf5c0ef9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:50.882570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:50.883294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:50.883574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:50.883895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:50.884661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:50.884966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:50.886309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d813-f11a-4978-8858-7dbfa1e1b955 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:50.898914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 534/1063] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:32:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:50.900046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:50.909531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506382-4b7f-4801-a3f3-c042e4d5bcbe None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:50.909823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506382-4b7f-4801-a3f3-c042e4d5bcbe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:50.910103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506382-4b7f-4801-a3f3-c042e4d5bcbe None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:50.911576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 535/1064] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:50.916387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:50.916680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:50.916963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:50.935355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:50.936576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:50.998938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaab92-f302-483f-9456-57654a6df354 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.002200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 536/1065] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:51.024527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.032663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.033461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.033747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.034023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.044795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.045960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:51.063303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:51.064255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:51.065203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:51.065203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:51.065203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:51.065203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:51.065203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 530/1066] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:51.070069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.133830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed980b8-44c7-45d0-bc48-ac99d9f5f05a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.172528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.173330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.173616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.173801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7bc33-e515-41b8-9879-8f8520d08be8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.173971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.184289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:51.193022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:51.194153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:51.195037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 537/1067] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:51.200899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.209831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.210495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.210782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.211057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.211774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:51.212081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.213466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.229831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 531/1068] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:51.230864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.300305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e765f-412a-45fe-aad4-1150f7e8dab4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.329235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762df5c0-a382-4d1f-9ef3-713a3fe0669a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.335642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.336339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.336626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.336900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.337643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:51.337949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.339305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7962-4ceb-461c-8121-3a18be5e1bf8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.347717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 538/1069] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:51.348965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667d5f-d351-470e-be10-faceb53c4e02 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.349257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667d5f-d351-470e-be10-faceb53c4e02 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.349535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667d5f-d351-470e-be10-faceb53c4e02 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.350922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 539/1070] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:51.353571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa667d5f-d351-470e-be10-faceb53c4e02 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.363921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.364740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.365020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.365288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.373027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.374456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ab245824-3f32-41d3-9d5d-2c68d1a23350 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:51.389258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:51.390375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:51.391656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 532/1071] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:51.392388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.392388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.392511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.408543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:51.409709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:51.452465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa667d5f-d351-470e-be10-faceb53c4e02 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.472628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 533/1072] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:51.473444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.489231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.490225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.490508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.490787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] PATH_INFO: `/v3/users/4d6c97d8b9d0411d80b7ada4244c3e77` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.492285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] RBAC: Authorizing `identity:get_user(user_id=4d6c97d8b9d0411d80b7ada4244c3e77)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.493764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.495531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 540/1073] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/users/4d6c97d8b9d0411d80b7ada4244c3e77 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:51.496155 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.569321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3a822-70eb-42d4-89bc-d373a2f08dec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.588971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030c4d0-43ed-4671-b3e5-f604704eb5a2 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.602376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.603044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.603326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.603599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.604462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:51.604770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.606733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.622280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.623161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.623447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.623717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.624174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 534/1074] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:51.625010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.693448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.694885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.716575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610df0b-ee03-464c-86b8-86e034f14796 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.748147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.749005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.749285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.749554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.750709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] RBAC: Authorizing `identity:get_project(project_id=6bbe7ea67e964b29ad51f1fb70c15f65)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.752111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.753807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 535/1075] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:51.754555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.795592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 541/1076] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:51.796623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.847089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269521e-2b94-4d56-a808-22615c1ee9cf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.880452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.881232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.881514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.881787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.889723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f366f-3674-4294-8d6a-ebbb09862e5f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:51.892413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.893847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:51.911541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:51.912506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:51.913406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:51.913406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:51.913406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:51.913406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:51.913406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 536/1077] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:51.913689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:51.923138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:51.924022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:51.924340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:51.924616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65/users/4d6c97d8b9d0411d80b7ada4244c3e77/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:51.926537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bbe7ea67e964b29ad51f1fb70c15f65, user_id=4d6c97d8b9d0411d80b7ada4244c3e77, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:51.927952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:51.940635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 542/1078] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:32:51 2022] PUT /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65/users/4d6c97d8b9d0411d80b7ada4244c3e77/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:32:51.941420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:52.027739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966beee-b303-40c9-bcd7-0e0da8f9fa67 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:52.058902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499df4-a14e-4550-997d-5c602de5b3e7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:52.065738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:52.066542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:52.066818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:52.067086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:52.078314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:52.079799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:52.096397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:52.097087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:52.097427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:52.097650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:52.098382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:52.098689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:52.098851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:52.099764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:52.101427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:52.101427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:52.101427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:52.101427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:52.101669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:52.101744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 537/1079] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:52.106595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:52.113220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 543/1080] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:52.119018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:52.204692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b08e6-c4d5-4921-b37b-8ec3f9bcad9d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:52.219077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8b63c-88bb-406b-a256-a43ed44914cc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:52.237741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:52.238505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:52.238791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:52.239063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:52.239778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:52.240081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:52.241733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:52.253359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 538/1081] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:52.253682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:52.254753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:52.255258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:52.255800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:52.263761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:52.265649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:52.281652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:52.282795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:52.283718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:52.288369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 544/1082] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:52.289846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:52.395580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6805-b473-4bcc-a7b4-db97d334fbdb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:52.431118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:52.431788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:52.432071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:52.432376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:52.433173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:52.433538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:52.436049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed016b2-d38d-4bb5-a0ae-799e2ddf90f3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:52.454304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 539/1083] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:52.455464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:52.558087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6eeb-7f34-410a-8a3d-8e259765cc08 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:52.592272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a23fec-dc1f-4a80-a636-9bc1e2cf7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:52.593149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a23fec-dc1f-4a80-a636-9bc1e2cf7597 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:52.593433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a23fec-dc1f-4a80-a636-9bc1e2cf7597 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:52.593713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a23fec-dc1f-4a80-a636-9bc1e2cf7597 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/62c3b98740d14ab49d91783086150e88/roles/c38c7f8d093b411e9b231ea8733760a4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:52.595881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a23fec-dc1f-4a80-a636-9bc1e2cf7597 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=62c3b98740d14ab49d91783086150e88, role_id=c38c7f8d093b411e9b231ea8733760a4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:52.597359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a23fec-dc1f-4a80-a636-9bc1e2cf7597 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:52.611220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 545/1084] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:32:52 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/62c3b98740d14ab49d91783086150e88/roles/c38c7f8d093b411e9b231ea8733760a4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:32:54.202957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaddc32-77d4-438f-892b-0bf2e83c58ad None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.203493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaddc32-77d4-438f-892b-0bf2e83c58ad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.203493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaddc32-77d4-438f-892b-0bf2e83c58ad None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.205156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 540/1085] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:54.210145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.210604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.210702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.227396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:54.228626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:54.294782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 546/1086] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:54.318556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaddc32-77d4-438f-892b-0bf2e83c58ad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:54.417277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaddc32-77d4-438f-892b-0bf2e83c58ad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:54.450558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:54.451471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.451750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.452021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.461813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:54.469989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:54.471107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:54.472019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 541/1087] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:32:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:54.477119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:54.572465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0d794-fc03-4791-9899-0a7c9ba62ce3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:54.605063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:54.605724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.606609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.607470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.608254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:54.608566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:54.609943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:54.618979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 547/1088] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:32:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:54.625449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:54.731874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dc6b5-adef-41b5-b82d-dfb033722f84 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:54.765195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40488e-c1e9-4c8f-8f76-6f21fb332e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:54.766058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40488e-c1e9-4c8f-8f76-6f21fb332e7b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.766344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40488e-c1e9-4c8f-8f76-6f21fb332e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.766625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40488e-c1e9-4c8f-8f76-6f21fb332e7b None admin] PATH_INFO: `/v3/users/4d6c97d8b9d0411d80b7ada4244c3e77` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.767798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40488e-c1e9-4c8f-8f76-6f21fb332e7b None admin] RBAC: Authorizing `identity:get_user(user_id=4d6c97d8b9d0411d80b7ada4244c3e77)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:54.769253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40488e-c1e9-4c8f-8f76-6f21fb332e7b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:54.770885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 542/1089] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:32:54 2022] GET /identity/v3/users/4d6c97d8b9d0411d80b7ada4244c3e77 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:54.776555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:54.813106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba26f50-bfd8-4713-8fff-19c4bea24032 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.813390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba26f50-bfd8-4713-8fff-19c4bea24032 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.813669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba26f50-bfd8-4713-8fff-19c4bea24032 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.815178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 543/1090] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:32:54.820529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.820856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.821141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.838444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:54.839622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:54.874109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e518c58-bb71-46bc-82e4-3ce4078c9671 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:54.905787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:54.905980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 544/1091] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:32:54.906427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:54.906706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:54.906988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:54.929760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:54.970670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:54.972101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.019487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466e4a2-d97b-4588-8fdc-2c3c60fb2af9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.050690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.051480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.051757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.052029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.061482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:32:55.063190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 548/1092] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:55.068942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:32:55.069489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:55.070545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:55.071911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 545/1093] 10.222.0.73 () {56 vars in 1120 bytes} [Thu May 12 17:32:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:32:55.076900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.161558 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18311143-0191-45e7-9957-b4f1050c9c3f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.169950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def4ac-a918-4100-b3ae-2a17eb2bae6f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.192499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.193347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.193625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.193894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.195052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] RBAC: Authorizing `identity:get_project(project_id=6bbe7ea67e964b29ad51f1fb70c15f65)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.196489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.198167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 549/1094] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:55.201448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.202107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.202388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.202727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.203472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:55.204011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.204561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.205147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.218139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 546/1095] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:55.219107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.296636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7cfd-b9ab-4ce1-91dc-428a3ac2a41d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.311412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c7682-b8e0-48fd-a857-0b43700c814e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.329002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.329643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.329930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.330202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.331023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65, user.id=4d6c97d8b9d0411d80b7ada4244c3e77) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:55.331329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.332781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.338388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 550/1096] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/role_assignments?user.id=4d6c97d8b9d0411d80b7ada4244c3e77&scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:32:55.343622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.344443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.344723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.344993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.356723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.358203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:55.376954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:55.378132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:55.384059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:55.384059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:55.384059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:55.384059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:55.384059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 547/1097] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:55.384419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.489675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccfadc-2607-4b22-8d9a-5a96b9143936 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.524544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.525209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.525498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.525762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.526476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:55.526778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.528321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.540052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 551/1098] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:55.546123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.642111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe2e6-316f-4c67-95c4-e30a966a8521 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.673678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.674457 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.674734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.675005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.682138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.683554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:32:55.697878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:55.698891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:32:55.700669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 548/1099] 10.222.0.73 () {56 vars in 1093 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:55.704876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.799508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579caa9f-7998-4a4b-b4d7-67980f1e8f4e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.831637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.832336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.832622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.832899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:55.833736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:55.834044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:55.835465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:55.851843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 552/1100] 10.222.0.73 () {56 vars in 1133 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:55.858130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:55.952956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710a5c8-9635-4586-9aac-2d135a2064a6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:55.983974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:55.984647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:55.984928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:55.985200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:56.050772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:56.052231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:56.150855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 549/1101] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:32:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:32:56.157298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:56.251246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0f4e3-c27e-4bf2-bc28-2fe97c66f5ce None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:56.282874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:56.283652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:56.283933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:56.284228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:56.294373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:56.295822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:32:56.312493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:56.313911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:56.314816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:32:56.314816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:32:56.314816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:32:56.314816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:56.314816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 553/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:56.319206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:56.416281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002eab-bd49-4e4a-81a1-8d6b8b2997f0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:56.448785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:56.449437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:56.450251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:56.450643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:56.451426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:56.451726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:56.453100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:56.463784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 550/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:56.469925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:56.564499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0c4c-80e5-46cc-9601-66f22c2a5047 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:56.597577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:56.598365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:56.598648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:56.598922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:56.605405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:56.607402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:32:56.621054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:32:56.622090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:32:56.622990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 554/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:32:56.628708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:56.725300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175605-9dd4-4f8f-b2ce-d9fd5a3796aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:56.756855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:56.757508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:56.757789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:56.758060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:56.758833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:56.759136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:56.760579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec60b3-5cc4-4953-b959-b306ec35c749 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:56.772476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 551/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:32:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:56.779063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:56.872825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c31837-963e-4fc2-8f57-e4d9dec632e3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:56.904507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:56.905151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:56.905430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:56.905700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:56.906524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] RBAC: Adding query filter params (role.id=c38c7f8d093b411e9b231ea8733760a4, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=62c3b98740d14ab49d91783086150e88) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:32:56.906831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:56.908261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d343c0f-614c-4822-b0e4-245b5e3d203c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:56.924202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 555/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:32:56 2022] GET /identity/v3/role_assignments?user.id=62c3b98740d14ab49d91783086150e88&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=c38c7f8d093b411e9b231ea8733760a4 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:57.592649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549551-8d05-4c22-aabb-5c4991cd6352 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:57.593214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549551-8d05-4c22-aabb-5c4991cd6352 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:57.593214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549551-8d05-4c22-aabb-5c4991cd6352 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:57.594559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 552/1107] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:32:57.600082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:57.600526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:57.600665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:57.620257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:32:57.621447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:32:57.687311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 556/1108] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:32:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:32:57.712512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39549551-8d05-4c22-aabb-5c4991cd6352 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:57.808984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39549551-8d05-4c22-aabb-5c4991cd6352 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:57.841707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de372e32-814f-4c0d-bd70-3ed9d2d5c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:57.842481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372e32-814f-4c0d-bd70-3ed9d2d5c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:57.842810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372e32-814f-4c0d-bd70-3ed9d2d5c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:57.843036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372e32-814f-4c0d-bd70-3ed9d2d5c1e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:57.844258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de372e32-814f-4c0d-bd70-3ed9d2d5c1e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:57.845681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de372e32-814f-4c0d-bd70-3ed9d2d5c1e4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:57.847281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 553/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:32:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:32:57.853045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:32:57.946620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6bc82-c269-4e86-8a9c-26479d36f600 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:32:57.978203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf358c33-3a5b-4bea-88bd-9a59d7ac934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:32:57.978854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358c33-3a5b-4bea-88bd-9a59d7ac934b None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:32:57.979137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358c33-3a5b-4bea-88bd-9a59d7ac934b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:32:57.979413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358c33-3a5b-4bea-88bd-9a59d7ac934b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:32:57.980107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf358c33-3a5b-4bea-88bd-9a59d7ac934b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:32:57.982553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf358c33-3a5b-4bea-88bd-9a59d7ac934b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:32:58.001558 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 557/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:32:57 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:33:00.243576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f67ea9-7205-49aa-a022-a6cc5ad551e4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:00.244141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f67ea9-7205-49aa-a022-a6cc5ad551e4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:00.244141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f67ea9-7205-49aa-a022-a6cc5ad551e4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:00.245501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 554/1111] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:00.251058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:00.251460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:00.251603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:00.268099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:00.269283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:00.330126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 558/1112] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:00.353299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f67ea9-7205-49aa-a022-a6cc5ad551e4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:00.446775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f67ea9-7205-49aa-a022-a6cc5ad551e4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:00.478988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:00.479807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:00.480085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:00.480381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] PATH_INFO: `/v3/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:00.481844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] RBAC: Authorizing `identity:get_role(role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:00.483183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:00.484545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 555/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:00 2022] GET /identity/v3/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 223 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:00.490228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:00.583673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d43e95-a325-4ef8-8413-2796f3c7c9dd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:00.615823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:00.616962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:00.617243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:00.617514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] PATH_INFO: `/v3/users/409467d4970242cd8c9de3b32150211a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:00.630975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] RBAC: Authorizing `identity:get_user(user_id=409467d4970242cd8c9de3b32150211a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:00.632432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:00.634067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 559/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:00 2022] GET /identity/v3/users/409467d4970242cd8c9de3b32150211a => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:00.640381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:00.730568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4c514-130c-424a-8fec-7d694457ae01 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:00.761943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:00.763113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:00.763386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:00.763659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:00.826737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:00.828185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:00.918913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 556/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:00.924902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:01.016959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec338-a5e9-460e-a67d-b9ee846da240 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:01.048822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c9cc5-f049-4fd0-b462-c482efbbfdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:01.049649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c9cc5-f049-4fd0-b462-c482efbbfdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:01.049928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c9cc5-f049-4fd0-b462-c482efbbfdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:01.050200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c9cc5-f049-4fd0-b462-c482efbbfdd0 None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:01.051350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c9cc5-f049-4fd0-b462-c482efbbfdd0 None admin] RBAC: Authorizing `identity:get_project(project_id=6bbe7ea67e964b29ad51f1fb70c15f65)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:01.052797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c9cc5-f049-4fd0-b462-c482efbbfdd0 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:01.054445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 560/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:00 2022] GET /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:01.061247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:01.151560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5d070-ce69-4be8-a785-c93f04bd753d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:01.186269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:01.186849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:01.187129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:01.187404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:01.188255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] RBAC: Adding query filter params (role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65, user.id=409467d4970242cd8c9de3b32150211a) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:01.188568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:01.189984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201a3a3-b704-40a8-abd1-bb3e79b9d00f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:01.195185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 557/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:33:01 2022] GET /identity/v3/role_assignments?user.id=409467d4970242cd8c9de3b32150211a&scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:33:03.398145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adb7d1-c2ab-4d66-9163-6e5290858634 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:03.398708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adb7d1-c2ab-4d66-9163-6e5290858634 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:03.398708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adb7d1-c2ab-4d66-9163-6e5290858634 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:03.400045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 561/1118] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:03.406055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:03.406474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:03.406609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:03.422622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:03.423791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:03.485673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 558/1119] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:03.508739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adb7d1-c2ab-4d66-9163-6e5290858634 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:03.606936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adb7d1-c2ab-4d66-9163-6e5290858634 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:03.640337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:03.641186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:03.641468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:03.641743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] PATH_INFO: `/v3/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:03.643196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] RBAC: Authorizing `identity:get_role(role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:03.644571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:03.645927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 562/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:03 2022] GET /identity/v3/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:03.651794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:03.743091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa0c6a-3d56-4938-a4b2-ddd4d31a6c1c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:03.774730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:03.775567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:03.775843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:03.776112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] PATH_INFO: `/v3/users/409467d4970242cd8c9de3b32150211a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:03.777282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] RBAC: Authorizing `identity:get_user(user_id=409467d4970242cd8c9de3b32150211a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:03.778890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:03.780511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 559/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:03 2022] GET /identity/v3/users/409467d4970242cd8c9de3b32150211a => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:03.786118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:03.878998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b02936-ad5d-4d60-90f8-0ad9aaebe49d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:03.910638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c57c1-6eae-4cf7-80eb-f26252ef0b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:03.911475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c57c1-6eae-4cf7-80eb-f26252ef0b0c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:03.911753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c57c1-6eae-4cf7-80eb-f26252ef0b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:03.912020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c57c1-6eae-4cf7-80eb-f26252ef0b0c None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:03.913205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c57c1-6eae-4cf7-80eb-f26252ef0b0c None admin] RBAC: Authorizing `identity:get_project(project_id=6bbe7ea67e964b29ad51f1fb70c15f65)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:03.914606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c57c1-6eae-4cf7-80eb-f26252ef0b0c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:03.916269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 563/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:03 2022] GET /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:03.922571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:04.014523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff8faf-af09-40bc-8df2-d41a4900868b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:04.045826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0abd44-3113-4f85-888e-601b0f3e9540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:04.046674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0abd44-3113-4f85-888e-601b0f3e9540 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:04.046947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0abd44-3113-4f85-888e-601b0f3e9540 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:04.047212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0abd44-3113-4f85-888e-601b0f3e9540 None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65/users/409467d4970242cd8c9de3b32150211a/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:04.049312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0abd44-3113-4f85-888e-601b0f3e9540 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bbe7ea67e964b29ad51f1fb70c15f65, user_id=409467d4970242cd8c9de3b32150211a, role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:04.050758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0abd44-3113-4f85-888e-601b0f3e9540 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:04.063565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 560/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:33:03 2022] PUT /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65/users/409467d4970242cd8c9de3b32150211a/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:33:06.265743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431568-4e20-48e3-b35c-1c405fee2d65 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:06.266279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431568-4e20-48e3-b35c-1c405fee2d65 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:06.266279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431568-4e20-48e3-b35c-1c405fee2d65 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:06.267654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 564/1124] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:06.272939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:06.273312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:06.273488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:06.289898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:06.291050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:06.352354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 561/1125] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:06.376947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12431568-4e20-48e3-b35c-1c405fee2d65 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:06.473149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12431568-4e20-48e3-b35c-1c405fee2d65 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:06.506155 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:06.506988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:06.507272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:06.507544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] PATH_INFO: `/v3/roles/4446567b2b5f417e9d55ae507cf04b48` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:06.509028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] RBAC: Authorizing `identity:get_role(role_id=4446567b2b5f417e9d55ae507cf04b48)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:06.510376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:06.511722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 565/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:06 2022] GET /identity/v3/roles/4446567b2b5f417e9d55ae507cf04b48 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:06.517970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:06.607959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02defc3-c3d7-48a9-81f3-511a0a264c92 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:06.639175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:06.640017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:06.640324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:06.640594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] PATH_INFO: `/v3/users/409467d4970242cd8c9de3b32150211a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:06.641741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] RBAC: Authorizing `identity:get_user(user_id=409467d4970242cd8c9de3b32150211a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:06.643154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:06.644790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 562/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:06 2022] GET /identity/v3/users/409467d4970242cd8c9de3b32150211a => generated 286 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:06.650395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:06.743699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0a3cd-b987-4626-9e5b-5d1f66892ec4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:06.774984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:06.775552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:06.775834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:06.776109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:06.838281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:06.839715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:06.930950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 566/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:06.937141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:07.030035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b4f4a-0b1b-439c-984b-831b211f6024 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:07.060463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee96639-9bd7-4210-833f-4f9348bbf543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:07.061293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee96639-9bd7-4210-833f-4f9348bbf543 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:07.061572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee96639-9bd7-4210-833f-4f9348bbf543 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:07.061839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee96639-9bd7-4210-833f-4f9348bbf543 None admin] PATH_INFO: `/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:07.062982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee96639-9bd7-4210-833f-4f9348bbf543 None admin] RBAC: Authorizing `identity:get_project(project_id=6bbe7ea67e964b29ad51f1fb70c15f65)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:07.064419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee96639-9bd7-4210-833f-4f9348bbf543 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:07.066065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 563/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:06 2022] GET /identity/v3/projects/6bbe7ea67e964b29ad51f1fb70c15f65 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:07.072251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:07.167551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb4976-8698-432f-b7d2-950942f329fc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:07.199321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:07.201545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:07.201833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:07.202111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:07.202935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] RBAC: Adding query filter params (role.id=4446567b2b5f417e9d55ae507cf04b48, scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65, user.id=409467d4970242cd8c9de3b32150211a) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:07.203248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:07.204691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779473-edf2-4f3a-8698-387bcf3082ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:07.210180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 567/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:33:07 2022] GET /identity/v3/role_assignments?user.id=409467d4970242cd8c9de3b32150211a&scope.project.id=6bbe7ea67e964b29ad51f1fb70c15f65&role.id=4446567b2b5f417e9d55ae507cf04b48 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:09.446768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413068-f678-4593-bf65-abc44cfb06a1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:09.447298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413068-f678-4593-bf65-abc44cfb06a1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:09.447298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413068-f678-4593-bf65-abc44cfb06a1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:09.448700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 564/1131] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:09.454357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:09.454781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:09.454900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:09.471340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:09.472542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:09.536021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 568/1132] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:33:09.559365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413068-f678-4593-bf65-abc44cfb06a1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:09.654667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413068-f678-4593-bf65-abc44cfb06a1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:09.685857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95538808-171f-4e6e-a188-46909f96a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:09.686621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95538808-171f-4e6e-a188-46909f96a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:09.686895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95538808-171f-4e6e-a188-46909f96a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:09.687162 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95538808-171f-4e6e-a188-46909f96a2f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:09.688324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95538808-171f-4e6e-a188-46909f96a2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:09.689732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95538808-171f-4e6e-a188-46909f96a2f6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:09.691309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 565/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:33:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:09.697210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:09.788146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924e3d1-f07f-42dd-b083-74ff0272d956 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:09.820319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de84b1-aa37-4f34-98cb-257ce5976b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:09.820960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de84b1-aa37-4f34-98cb-257ce5976b5e None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:09.821239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de84b1-aa37-4f34-98cb-257ce5976b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:09.821508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de84b1-aa37-4f34-98cb-257ce5976b5e None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:09.822195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de84b1-aa37-4f34-98cb-257ce5976b5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:09.823641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de84b1-aa37-4f34-98cb-257ce5976b5e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:09.848405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 569/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:33:09 2022] POST /identity/v3/projects => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:33:12.163721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93bd33-9416-4e4c-8780-d115eddac449 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:12.164350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93bd33-9416-4e4c-8780-d115eddac449 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:12.164482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93bd33-9416-4e4c-8780-d115eddac449 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:12.165854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 566/1135] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:12.172095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:12.172557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:12.172674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:12.189416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:12.190581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:12.252368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 570/1136] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:12.276527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93bd33-9416-4e4c-8780-d115eddac449 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:12.370020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93bd33-9416-4e4c-8780-d115eddac449 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:12.401277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6058be-f67c-497b-a504-8571d13482e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:12.402037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6058be-f67c-497b-a504-8571d13482e7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:12.402312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6058be-f67c-497b-a504-8571d13482e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:12.402601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6058be-f67c-497b-a504-8571d13482e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:12.403744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6058be-f67c-497b-a504-8571d13482e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:12.405164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6058be-f67c-497b-a504-8571d13482e7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:12.406729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 567/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:33:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:12.412295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:12.505318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fad7a-96db-49ba-a9c5-938b7f892545 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:12.536990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc67cac-5fc1-4c82-bfa1-8c14c7c99472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:12.537641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc67cac-5fc1-4c82-bfa1-8c14c7c99472 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:12.537924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc67cac-5fc1-4c82-bfa1-8c14c7c99472 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:12.538195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc67cac-5fc1-4c82-bfa1-8c14c7c99472 None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:12.538897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc67cac-5fc1-4c82-bfa1-8c14c7c99472 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:12.540349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc67cac-5fc1-4c82-bfa1-8c14c7c99472 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:12.559438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 571/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:33:12 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:33:14.784663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4b2f-9c3a-48c1-b44d-f42411835a7f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:14.785218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4b2f-9c3a-48c1-b44d-f42411835a7f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:14.785218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b4b2f-9c3a-48c1-b44d-f42411835a7f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:14.786551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 568/1139] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:14.791699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:14.792062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:14.792382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:14.808375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:14.810165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:14.874890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 572/1140] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:14.899400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4b2f-9c3a-48c1-b44d-f42411835a7f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:14.997874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b4b2f-9c3a-48c1-b44d-f42411835a7f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:15.030120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:15.030889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:15.031172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:15.031446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:15.040926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:33:15.048983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:15.050081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:33:15.051130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 569/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:15.056008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:15.146873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61409286-a0d0-422d-bcbf-6eb79b95a5c2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:15.188277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:15.188947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:15.189231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:15.189506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:15.190255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:15.190562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:15.191940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:15.199994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 573/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:33:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:15.205770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:15.299330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847fedc-a470-4308-b2ab-89a3d18dabf7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:15.330295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:15.331141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:15.331420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:15.331689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] PATH_INFO: `/v3/users/653a383cdead4f2b85406d4944620fbb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:15.343989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] RBAC: Authorizing `identity:get_user(user_id=653a383cdead4f2b85406d4944620fbb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:15.345437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:15.347042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 570/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:15 2022] GET /identity/v3/users/653a383cdead4f2b85406d4944620fbb => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:15.352755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:15.445901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795fd67-9389-4996-9413-a3fce18d9ecc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:15.476117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:15.476771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:15.477054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:15.477327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:15.542218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:15.544245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:15.634980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 574/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:15.641051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:15.735200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c2e1b-6985-4c27-b0a8-c9f4893bf438 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:15.766485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874df16-4841-4cd8-9030-08dab0155b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:15.767316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874df16-4841-4cd8-9030-08dab0155b4a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:15.767600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874df16-4841-4cd8-9030-08dab0155b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:15.767873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874df16-4841-4cd8-9030-08dab0155b4a None admin] PATH_INFO: `/v3/projects/01e85fd1aac64a6b917707d74f5c40ba` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:15.769062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874df16-4841-4cd8-9030-08dab0155b4a None admin] RBAC: Authorizing `identity:get_project(project_id=01e85fd1aac64a6b917707d74f5c40ba)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:15.770500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874df16-4841-4cd8-9030-08dab0155b4a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:15.772175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 571/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:15 2022] GET /identity/v3/projects/01e85fd1aac64a6b917707d74f5c40ba => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:15.778597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:15.869156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6dc9a-1ca3-42fb-9788-b22644a496ee None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:15.899560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:15.900208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:15.900489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:15.900758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:15.901577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=01e85fd1aac64a6b917707d74f5c40ba, user.id=653a383cdead4f2b85406d4944620fbb) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:15.901875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:15.903781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219916d2-edf2-449e-a00b-cb6580087a55 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:15.908904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 575/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:33:15 2022] GET /identity/v3/role_assignments?user.id=653a383cdead4f2b85406d4944620fbb&scope.project.id=01e85fd1aac64a6b917707d74f5c40ba&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:18.156051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e07bda-eb4a-49c5-a4b1-6d3174985d98 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.156801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e07bda-eb4a-49c5-a4b1-6d3174985d98 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.156801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e07bda-eb4a-49c5-a4b1-6d3174985d98 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.157981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 572/1147] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:18.163391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.163805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.163939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.180606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:18.181776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:18.255454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 576/1148] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:18.278159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e07bda-eb4a-49c5-a4b1-6d3174985d98 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:18.371942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e07bda-eb4a-49c5-a4b1-6d3174985d98 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:18.402577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:18.403345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.403619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.403889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.413337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:33:18.421506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:18.422703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:33:18.423588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 573/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:33:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:18.428582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:18.523004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e265-136b-4c7a-bd8a-a8ea59abdb9c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:18.557701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:18.558365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.558677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.559425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.560197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:18.560512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:18.561878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:18.569489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 577/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:18.575629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:18.671348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c07a4-e7c4-4403-86fd-da423a4a916d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:18.702760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:18.703606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.703885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.704206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] PATH_INFO: `/v3/users/653a383cdead4f2b85406d4944620fbb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.705377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] RBAC: Authorizing `identity:get_user(user_id=653a383cdead4f2b85406d4944620fbb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:18.706794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:18.708438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 574/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:18 2022] GET /identity/v3/users/653a383cdead4f2b85406d4944620fbb => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:18.714123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:18.810351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3502e5-2e12-4d9c-b870-9589348f84db None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:18.842242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347919f9-b79d-45d7-b070-c65d97ae21dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:18.843078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347919f9-b79d-45d7-b070-c65d97ae21dd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.843363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347919f9-b79d-45d7-b070-c65d97ae21dd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.843638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347919f9-b79d-45d7-b070-c65d97ae21dd None admin] PATH_INFO: `/v3/projects/01e85fd1aac64a6b917707d74f5c40ba` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.844799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347919f9-b79d-45d7-b070-c65d97ae21dd None admin] RBAC: Authorizing `identity:get_project(project_id=01e85fd1aac64a6b917707d74f5c40ba)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:18.846202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347919f9-b79d-45d7-b070-c65d97ae21dd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:18.847852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 578/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:18 2022] GET /identity/v3/projects/01e85fd1aac64a6b917707d74f5c40ba => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:18.853499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:18.949035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69616f59-6a28-46f4-b3a5-70e3beae88ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:18.981887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8349-24af-4896-8a7e-a23da3d2886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:18.982736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8349-24af-4896-8a7e-a23da3d2886a None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:18.983019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8349-24af-4896-8a7e-a23da3d2886a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:18.983294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8349-24af-4896-8a7e-a23da3d2886a None admin] PATH_INFO: `/v3/projects/01e85fd1aac64a6b917707d74f5c40ba/users/653a383cdead4f2b85406d4944620fbb/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:18.985253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8349-24af-4896-8a7e-a23da3d2886a None admin] RBAC: Authorizing `identity:create_grant(project_id=01e85fd1aac64a6b917707d74f5c40ba, user_id=653a383cdead4f2b85406d4944620fbb, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:18.986664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8349-24af-4896-8a7e-a23da3d2886a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:19.000429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 575/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:33:18 2022] PUT /identity/v3/projects/01e85fd1aac64a6b917707d74f5c40ba/users/653a383cdead4f2b85406d4944620fbb/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:33:21.177446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ff477-280b-42eb-8873-a0d1b29a1eb7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:21.178033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ff477-280b-42eb-8873-a0d1b29a1eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:21.178033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ff477-280b-42eb-8873-a0d1b29a1eb7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:21.179332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 579/1154] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:21.184631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:21.185059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:21.185187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:21.203054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:21.204269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:21.268349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 576/1155] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:21.292036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ff477-280b-42eb-8873-a0d1b29a1eb7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:21.386733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ff477-280b-42eb-8873-a0d1b29a1eb7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:21.417200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:21.418485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:21.418775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:21.419130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:21.428646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:33:21.436849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:21.437979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:33:21.439200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 580/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:33:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:21.443931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:21.533636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d3246-2656-4ccf-aba5-aaa92e7876dd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:21.564424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:21.565060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:21.565337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:21.565606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:21.566335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:21.566633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:21.567967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:21.575595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 577/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:33:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:21.581226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:21.676460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768a366-8b21-4ea7-85c4-842c51e6f7f8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:21.707107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:21.707897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:21.708198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:21.708475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] PATH_INFO: `/v3/users/653a383cdead4f2b85406d4944620fbb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:21.709632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=653a383cdead4f2b85406d4944620fbb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:21.711041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:21.712659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 581/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:21 2022] GET /identity/v3/users/653a383cdead4f2b85406d4944620fbb => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:21.718931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:21.810730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7850ac-2416-4c16-8e9e-0f0522b0cfee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:21.842450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:21.843081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:21.843360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:21.843633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:21.905466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:21.906881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:21.996602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 578/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:22.002359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:22.095228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9711b0a-6d80-4ff7-aa01-5f013441e1b4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:22.126172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eddd2-46f5-4337-b402-9828b96829d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:22.127011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eddd2-46f5-4337-b402-9828b96829d1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:22.127293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eddd2-46f5-4337-b402-9828b96829d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:22.127564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eddd2-46f5-4337-b402-9828b96829d1 None admin] PATH_INFO: `/v3/projects/01e85fd1aac64a6b917707d74f5c40ba` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:22.128738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2eddd2-46f5-4337-b402-9828b96829d1 None admin] RBAC: Authorizing `identity:get_project(project_id=01e85fd1aac64a6b917707d74f5c40ba)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:22.130141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2eddd2-46f5-4337-b402-9828b96829d1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:22.131788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 582/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:22 2022] GET /identity/v3/projects/01e85fd1aac64a6b917707d74f5c40ba => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:22.138349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:22.229439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e75d6e-7573-4cf6-8847-155d9079bbf4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:22.261393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:22.262143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:22.262498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:22.262769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:22.263582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=01e85fd1aac64a6b917707d74f5c40ba, user.id=653a383cdead4f2b85406d4944620fbb) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:22.263886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:22.265312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e253-5425-4fc9-b481-5051e5ac6471 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:22.270753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 579/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:33:22 2022] GET /identity/v3/role_assignments?user.id=653a383cdead4f2b85406d4944620fbb&scope.project.id=01e85fd1aac64a6b917707d74f5c40ba&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:24.501811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0a5d-6268-4ef5-ba46-f62920e97388 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:24.502340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0a5d-6268-4ef5-ba46-f62920e97388 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:24.502340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0a5d-6268-4ef5-ba46-f62920e97388 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:24.503717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 583/1162] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:24.514424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:24.514841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:24.514970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:24.532292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:24.533441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:24.593993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 580/1163] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:33:24.615793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a0a5d-6268-4ef5-ba46-f62920e97388 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:24.709544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a0a5d-6268-4ef5-ba46-f62920e97388 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:24.740608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:24.741238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:24.741762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:24.742032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:24.807284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:24.809193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:24.899546 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 584/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:24.906331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:24.996614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d92346-3f7f-494a-a205-99eff6a9846a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:25.027255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:25.028591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:25.028862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:25.029125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:25.035347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:25.036783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f64991d8-7556-4d13-acf2-26621c1126a1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:25.050170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:25.051356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:33:25.052249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 581/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:33:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:25.057267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:25.149564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a8406-f875-49a9-8ba4-d33de13f32de None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:25.181079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:25.181723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:25.182004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:25.182276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:25.182984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:25.183285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:25.184715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab20f3-100e-4a9d-a1be-cea465086246 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:25.191466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 585/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Thu May 12 17:33:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:27.383830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9103fb-9fb8-4d37-bf6b-8dd03e975602 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:27.384449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9103fb-9fb8-4d37-bf6b-8dd03e975602 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:27.384449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9103fb-9fb8-4d37-bf6b-8dd03e975602 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:27.385749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 582/1167] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:27.391431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce17f3-e1b2-4c92-8d39-2d67afa19029 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:27.391830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce17f3-e1b2-4c92-8d39-2d67afa19029 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:27.391975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce17f3-e1b2-4c92-8d39-2d67afa19029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:27.408561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7cce17f3-e1b2-4c92-8d39-2d67afa19029 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:27.409716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7cce17f3-e1b2-4c92-8d39-2d67afa19029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:27.471220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 586/1168] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:27.493682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9103fb-9fb8-4d37-bf6b-8dd03e975602 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:27.589144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9103fb-9fb8-4d37-bf6b-8dd03e975602 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:27.620545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e692adb-90e3-4219-bc00-12e21fa0c032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:27.621696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e692adb-90e3-4219-bc00-12e21fa0c032 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:27.621973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e692adb-90e3-4219-bc00-12e21fa0c032 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:27.622248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e692adb-90e3-4219-bc00-12e21fa0c032 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:27.622972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e692adb-90e3-4219-bc00-12e21fa0c032 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:27.624349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e692adb-90e3-4219-bc00-12e21fa0c032 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:27.638339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 583/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Thu May 12 17:33:27 2022] POST /identity/v3/domains => generated 267 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:33:29.838132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c116ce3-c2b8-4a22-a10b-ceb43040d748 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:29.838674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c116ce3-c2b8-4a22-a10b-ceb43040d748 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:29.838674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c116ce3-c2b8-4a22-a10b-ceb43040d748 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:29.840059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 587/1170] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:29.845538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:29.845952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:29.846095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:29.862410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:29.863581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:29.925484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 584/1171] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:29.948815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c116ce3-c2b8-4a22-a10b-ceb43040d748 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:30.043157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c116ce3-c2b8-4a22-a10b-ceb43040d748 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:30.075447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879bfca-f4da-4634-a214-316eeb5a3144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:30.076302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879bfca-f4da-4634-a214-316eeb5a3144 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:30.076580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879bfca-f4da-4634-a214-316eeb5a3144 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:30.076847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879bfca-f4da-4634-a214-316eeb5a3144 None admin] PATH_INFO: `/v3/domains/9145b2d952494e318253bad57b053eff` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:30.086549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879bfca-f4da-4634-a214-316eeb5a3144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9145b2d952494e318253bad57b053eff)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:30.087973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879bfca-f4da-4634-a214-316eeb5a3144 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:30.089595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 588/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 12 17:33:29 2022] GET /identity/v3/domains/9145b2d952494e318253bad57b053eff => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:30.095235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:30.186743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48a0ca-596d-4b00-8242-27d1b79dbd93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:30.217430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c003671-1690-4628-9a62-01b85c0487cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:30.218081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003671-1690-4628-9a62-01b85c0487cc None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:30.218364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003671-1690-4628-9a62-01b85c0487cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:30.218627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c003671-1690-4628-9a62-01b85c0487cc None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:30.219328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c003671-1690-4628-9a62-01b85c0487cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:30.220807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c003671-1690-4628-9a62-01b85c0487cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:30.245462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 585/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:33:30 2022] POST /identity/v3/projects => generated 361 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:33:32.497189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbc0a5-f5e1-4fd2-9e3d-81fbea720d36 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:32.497743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbc0a5-f5e1-4fd2-9e3d-81fbea720d36 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:32.497858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbc0a5-f5e1-4fd2-9e3d-81fbea720d36 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:32.499226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 589/1174] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:32.504463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:32.504875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:32.505025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:32.522626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:32.523798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:32.588711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 586/1175] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:32.614086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbc0a5-f5e1-4fd2-9e3d-81fbea720d36 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:32.709150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbc0a5-f5e1-4fd2-9e3d-81fbea720d36 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:32.741134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabb631-648d-4944-b27a-1fa8e892bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:32.742245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb631-648d-4944-b27a-1fa8e892bff8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:32.742526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb631-648d-4944-b27a-1fa8e892bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:32.742799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabb631-648d-4944-b27a-1fa8e892bff8 None admin] PATH_INFO: `/v3/domains/9145b2d952494e318253bad57b053eff` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:32.743964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabb631-648d-4944-b27a-1fa8e892bff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9145b2d952494e318253bad57b053eff)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:32.745410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabb631-648d-4944-b27a-1fa8e892bff8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:32.747031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 590/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Thu May 12 17:33:32 2022] GET /identity/v3/domains/9145b2d952494e318253bad57b053eff => generated 267 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:32.752796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:32.844391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0f149-1c92-41ca-a25e-88289f27906f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:32.876540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b60229-f007-4796-ae16-7cb5ea13157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:32.877200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b60229-f007-4796-ae16-7cb5ea13157b None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:32.877483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b60229-f007-4796-ae16-7cb5ea13157b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:32.877756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b60229-f007-4796-ae16-7cb5ea13157b None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:32.878458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b60229-f007-4796-ae16-7cb5ea13157b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:32.879881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b60229-f007-4796-ae16-7cb5ea13157b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:32.900026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 587/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:33:32 2022] POST /identity/v3/users => generated 311 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:33:35.207046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0d8c-0c89-4f9a-8a0a-fc558fd3351e None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:35.207582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0d8c-0c89-4f9a-8a0a-fc558fd3351e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:35.207582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0d8c-0c89-4f9a-8a0a-fc558fd3351e None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:35.208957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 591/1178] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:35.214227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:35.215170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:35.215280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:35.231855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:35.233046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:35.295998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 588/1179] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:35.319250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af0d8c-0c89-4f9a-8a0a-fc558fd3351e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:35.413777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af0d8c-0c89-4f9a-8a0a-fc558fd3351e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:35.444815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:35.445578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:35.445857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:35.446129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:35.456224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:33:35.463596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:35.465400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:33:35.466494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 592/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:33:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:35.471746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:35.565338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b04239-7f69-4b34-b73f-407226ddc16b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:35.598418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:35.599073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:35.599353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:35.599626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:35.600399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:35.600706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:35.602061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:35.610253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 589/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:33:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:35.615901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:35.707665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26085ae1-002b-454c-b13d-f2ee149eae93 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:35.738889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:35.739662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:35.739941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:35.740552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] PATH_INFO: `/v3/users/fe648b8042a34b9c872faf1c0e2be54a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:35.752954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=fe648b8042a34b9c872faf1c0e2be54a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:35.754361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:35.755945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 593/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:35 2022] GET /identity/v3/users/fe648b8042a34b9c872faf1c0e2be54a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:35.762099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:35.858161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0335c89-0f51-4d6f-b38c-04389be25273 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:35.889650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:35.890275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:35.890552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:35.890821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:35.959507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:35.960964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:36.053295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 590/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:36.060053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:36.155800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ede455-6bce-4e4d-ba40-aeb49df8d1d4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:36.188621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c2c6a-7eef-4fe4-9498-20524c6e35da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:36.189459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c2c6a-7eef-4fe4-9498-20524c6e35da None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:36.189743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c2c6a-7eef-4fe4-9498-20524c6e35da None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:36.190016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c2c6a-7eef-4fe4-9498-20524c6e35da None admin] PATH_INFO: `/v3/projects/bbde96daca1045fa8be14e037ca680ac` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:36.191185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c2c6a-7eef-4fe4-9498-20524c6e35da None admin] RBAC: Authorizing `identity:get_project(project_id=bbde96daca1045fa8be14e037ca680ac)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:36.192637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c2c6a-7eef-4fe4-9498-20524c6e35da None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:36.194300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 594/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:36 2022] GET /identity/v3/projects/bbde96daca1045fa8be14e037ca680ac => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:36.200454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:36.298949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ec3cc-70f0-4d19-8115-c53f8183a90a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:36.331737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:36.332339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:36.332622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:36.332892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:36.333708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=bbde96daca1045fa8be14e037ca680ac, user.id=fe648b8042a34b9c872faf1c0e2be54a) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:36.334014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:36.335415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dd3ca-0181-4ccf-88a1-532bc7c4857c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:36.340569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 591/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:33:36 2022] GET /identity/v3/role_assignments?user.id=fe648b8042a34b9c872faf1c0e2be54a&scope.project.id=bbde96daca1045fa8be14e037ca680ac&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:38.566754 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a0bd-81c7-4754-ae38-f8f58429cf82 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:38.567285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a0bd-81c7-4754-ae38-f8f58429cf82 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:38.567285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a0bd-81c7-4754-ae38-f8f58429cf82 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:38.568711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 595/1186] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:38.577352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:38.578361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:38.578361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:38.594695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:38.595887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:38.658228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 592/1187] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:33:38.680294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99a0bd-81c7-4754-ae38-f8f58429cf82 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:38.777226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99a0bd-81c7-4754-ae38-f8f58429cf82 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:38.810126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:38.811212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:38.811503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:38.811782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:38.821601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:33:38.830174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:38.831280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:33:38.832423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 596/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:33:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:38.837239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:38.931225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be1784-883a-4fcd-b1c4-7ac204194545 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:38.962553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:38.963207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:38.963488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:38.963760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:38.964524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:38.964831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:38.966758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:38.974485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 593/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:33:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:38.980399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:39.072856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afb13b-9693-478a-9564-bf0bc6f5c84f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:39.103658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:39.104529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:39.104813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:39.105087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] PATH_INFO: `/v3/users/fe648b8042a34b9c872faf1c0e2be54a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:39.106250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] RBAC: Authorizing `identity:get_user(user_id=fe648b8042a34b9c872faf1c0e2be54a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:39.108394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:39.109977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 597/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:38 2022] GET /identity/v3/users/fe648b8042a34b9c872faf1c0e2be54a => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:39.115507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:39.214439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938ea99-dcc6-4686-ad54-a4ba936ff75f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:39.246122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a93e6-a492-4c55-9b7e-64479b3fef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:39.246986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a93e6-a492-4c55-9b7e-64479b3fef4f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:39.247265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a93e6-a492-4c55-9b7e-64479b3fef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:39.247526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a93e6-a492-4c55-9b7e-64479b3fef4f None admin] PATH_INFO: `/v3/projects/bbde96daca1045fa8be14e037ca680ac` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:39.248686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a93e6-a492-4c55-9b7e-64479b3fef4f None admin] RBAC: Authorizing `identity:get_project(project_id=bbde96daca1045fa8be14e037ca680ac)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:39.250098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a93e6-a492-4c55-9b7e-64479b3fef4f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:39.251763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 594/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:39 2022] GET /identity/v3/projects/bbde96daca1045fa8be14e037ca680ac => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:39.257791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:39.350825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455a4a-633c-4f64-85a0-20ac2010ecd7 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:39.381072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b582c-235a-4b38-b09e-75777381471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:39.381922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b582c-235a-4b38-b09e-75777381471a None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:39.382197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b582c-235a-4b38-b09e-75777381471a None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:39.382466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b582c-235a-4b38-b09e-75777381471a None admin] PATH_INFO: `/v3/projects/bbde96daca1045fa8be14e037ca680ac/users/fe648b8042a34b9c872faf1c0e2be54a/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:39.384383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b582c-235a-4b38-b09e-75777381471a None admin] RBAC: Authorizing `identity:create_grant(project_id=bbde96daca1045fa8be14e037ca680ac, user_id=fe648b8042a34b9c872faf1c0e2be54a, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:39.385779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b582c-235a-4b38-b09e-75777381471a None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:39.398322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 598/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:33:39 2022] PUT /identity/v3/projects/bbde96daca1045fa8be14e037ca680ac/users/fe648b8042a34b9c872faf1c0e2be54a/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:33:41.571689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54710-402f-4f4a-a3dd-406faf323396 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:41.572261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54710-402f-4f4a-a3dd-406faf323396 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:41.572261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e54710-402f-4f4a-a3dd-406faf323396 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:41.573612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 595/1193] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:33:41.579549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:41.580012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:41.580106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:41.596826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:33:41.597975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:33:41.664712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 599/1194] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:33:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:33:41.689792 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54710-402f-4f4a-a3dd-406faf323396 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:41.786643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e54710-402f-4f4a-a3dd-406faf323396 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:41.818067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:41.818937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:41.819304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:41.819723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:41.830614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:33:41.838068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:33:41.839127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:33:41.840292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 596/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:33:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:33:41.845739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:41.938396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3ca8-c389-463a-9bb0-f1b5a81282bd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:41.969544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:41.970196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:41.970478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:41.970753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:41.971492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:41.971794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:41.973169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:41.980701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 600/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:33:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:41.987213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:42.079605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ea234-d864-4db5-b1eb-0bc2f2a2debc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:42.111869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:42.112681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:42.112959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:42.113227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] PATH_INFO: `/v3/users/fe648b8042a34b9c872faf1c0e2be54a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:42.114372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] RBAC: Authorizing `identity:get_user(user_id=fe648b8042a34b9c872faf1c0e2be54a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:42.115772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:42.117639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 597/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Thu May 12 17:33:41 2022] GET /identity/v3/users/fe648b8042a34b9c872faf1c0e2be54a => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:42.123206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:42.217891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde08cd-c6d2-4d60-a019-53bcf48cadb4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:42.250798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:42.251440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:42.251724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:42.251997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:42.315081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:42.316540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:42.408216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 601/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:33:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:33:42.414060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:42.509952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06775604-e933-49b3-befe-cf0152cf729d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:42.542986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2c130-48ac-4228-b3ff-4e808ecd6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:42.544629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2c130-48ac-4228-b3ff-4e808ecd6f93 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:42.544911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2c130-48ac-4228-b3ff-4e808ecd6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:42.545178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2c130-48ac-4228-b3ff-4e808ecd6f93 None admin] PATH_INFO: `/v3/projects/bbde96daca1045fa8be14e037ca680ac` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:42.546339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2c130-48ac-4228-b3ff-4e808ecd6f93 None admin] RBAC: Authorizing `identity:get_project(project_id=bbde96daca1045fa8be14e037ca680ac)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:42.547749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2c130-48ac-4228-b3ff-4e808ecd6f93 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:42.549468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 598/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Thu May 12 17:33:42 2022] GET /identity/v3/projects/bbde96daca1045fa8be14e037ca680ac => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:33:42.555616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:33:42.649524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b65150-0127-4fb2-881f-7a1116a8974b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:33:42.682563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:33:42.683144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:33:42.683428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:33:42.683704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:33:42.684901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=bbde96daca1045fa8be14e037ca680ac, user.id=fe648b8042a34b9c872faf1c0e2be54a) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:33:42.685318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:33:42.686745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d10a4f-596e-4d55-878a-88a8f355727d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:33:42.692304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 602/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:33:42 2022] GET /identity/v3/role_assignments?user.id=fe648b8042a34b9c872faf1c0e2be54a&scope.project.id=bbde96daca1045fa8be14e037ca680ac&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:43.429309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b8615-339e-46c4-84b5-e2c2b796468b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:43.429893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b8615-339e-46c4-84b5-e2c2b796468b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:43.429893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b8615-339e-46c4-84b5-e2c2b796468b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:43.431234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 599/1201] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:34:43.437808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:43.438334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:43.438572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:43.456255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:34:43.457783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:34:43.531833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 603/1202] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:34:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:34:43.557042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b8615-339e-46c4-84b5-e2c2b796468b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:43.680525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b8615-339e-46c4-84b5-e2c2b796468b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:43.719157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:43.720091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:43.720526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:43.720807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:43.734094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:43.734752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:43.758836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:43.760006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:43.761757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:43.761757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:43.761757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:43.761757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:34:43.761757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 600/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:43.766565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:43.885232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c840a45-e791-4ba3-b259-2d4190f88c7c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:43.930235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:43.931003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:43.931283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:43.931529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:43.932428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:43.932642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:43.935364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028438bf-8911-43b3-be37-c4794a890888 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:43.953745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 604/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:43.955746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:44.088593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ebf95-2a01-4e20-9cb2-b5b686f074df None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:44.133181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fefac6-596d-42d0-947f-0dacb15b057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:44.133864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fefac6-596d-42d0-947f-0dacb15b057c None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:44.134149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fefac6-596d-42d0-947f-0dacb15b057c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:44.134426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fefac6-596d-42d0-947f-0dacb15b057c None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:44.135130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fefac6-596d-42d0-947f-0dacb15b057c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:44.136606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fefac6-596d-42d0-947f-0dacb15b057c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:44.168406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 601/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Thu May 12 17:34:43 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:34:46.632002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ddd9-9319-4181-8315-2e0a0b27c6e1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:46.632656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ddd9-9319-4181-8315-2e0a0b27c6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:46.632656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6ddd9-9319-4181-8315-2e0a0b27c6e1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:46.634163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 605/1206] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:34:46.641597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:46.642164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:46.642287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:46.662699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:34:46.664007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:34:46.748128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 602/1207] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:34:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:34:46.771763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6ddd9-9319-4181-8315-2e0a0b27c6e1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:46.901230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6ddd9-9319-4181-8315-2e0a0b27c6e1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:46.940896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:46.941698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:46.942045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:46.942299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:46.955152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:34:46.965345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:46.966395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:46.967245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 606/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:34:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:46.973106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:47.084766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b904a-7eea-4bb5-96b1-702e37aa4f81 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:47.123910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:47.124605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:47.124884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:47.125155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:47.125893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:47.126253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:47.127759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:47.136016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 603/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:34:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:47.141971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:47.261972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b51d4-6cc6-446c-9443-b0c73c79af5d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:47.298150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:47.299003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:47.299293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:47.299571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:47.311311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:47.312948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:47.331964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:47.333091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:47.334075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:47.334075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:47.334075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:47.334075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:47.334075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 607/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:47.339419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:47.452583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091bcb9-eb9e-4d08-8979-4b634862a10a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:47.487871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:47.488568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:47.488853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:47.489133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:47.489906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:47.490207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:47.491568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:47.504043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 604/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:47.509692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:47.625222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebe40d-4e8f-46e7-869c-d6da2fd0798e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:47.664689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:47.665513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:47.665801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:47.666079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:47.675782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:47.675782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:34:47.690647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:47.691844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:47.692862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 608/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:34:47.698856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:47.822734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85add71-b2ae-4deb-ade7-99cf23e12ccf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:47.868266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:47.869031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:47.869444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:47.869883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:47.871737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:47.872118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:47.873800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:47.895364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 605/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 12 17:34:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:47.901129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:48.017152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c8e88-3d11-4b76-9da5-5b366a62dea4 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:48.059975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:48.060933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:48.061240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:48.061566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:48.140050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:48.141792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:48.258361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 609/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:34:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:34:48.259939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:48.376797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29af5cd-6200-4e54-a95a-a6dcbe1d0242 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:48.412002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:48.412825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:48.413106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:48.413380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:48.424283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:48.426170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:48.446067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:48.453386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:48.454348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:48.454348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:48.454348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:48.454348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:34:48.454348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 606/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:48.454639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:48.563065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9d419-ccac-4d52-a65a-cb2612ea60bb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:48.601915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:48.602620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:48.602958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:48.603241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:48.603982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:48.604335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:48.606371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:48.617446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 610/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:34:48.624411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:48.726892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303e37-da62-4328-9c68-b12a20a95ce3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:48.765334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:48.766131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:48.766407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:48.766679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:48.773318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:48.774751 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:34:48.791061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:48.792201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:34:48.793289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:34:48.795376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 607/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:48.801004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:48.908623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567b460-1226-4fc5-9c93-9c26d4b8e2c9 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:48.944858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:48.945979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:48.946481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:48.946950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:48.948057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:48.948720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:48.950883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c20205-1256-4c9d-a79a-4a794b23e0b6 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:48.964213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 611/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:48.970425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:49.083367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c86b09-e2d7-4b5c-84f3-61821a70eb6a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:49.118676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:49.119494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:49.119800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:49.120090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:49.121906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3704ba4cfa7f4ad8b798636b01ebd95b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:49.122354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:49.123991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fa02a-75ce-471d-9e8c-ed894a565298 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:49.130570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 608/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:34:48 2022] GET /identity/v3/role_assignments?user.id=3704ba4cfa7f4ad8b798636b01ebd95b&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:34:51.478737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1d66-600c-470a-8ce7-1a637dc94fa5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:51.479285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1d66-600c-470a-8ce7-1a637dc94fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:51.479285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1d66-600c-470a-8ce7-1a637dc94fa5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:51.481341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 612/1220] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:34:51.486662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:51.487066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:51.487229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:51.506702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:34:51.507971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:34:51.577625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 609/1221] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:34:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:34:51.600106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1d66-600c-470a-8ce7-1a637dc94fa5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:51.712625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1d66-600c-470a-8ce7-1a637dc94fa5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:51.748698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:51.749419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:51.749705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:51.749980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:51.763754 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:34:51.772308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:51.773292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:34:51.774170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:51.778777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 613/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:34:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:51.779977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:51.886141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a99201-b2ff-44c8-af63-d7a808cdf497 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:51.922372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:51.923047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:51.923380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:51.923653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:51.924471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:51.924778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:51.926129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:51.935650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 610/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:34:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:51.941342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:52.048251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6acdf2-f8c3-4aac-8ef8-0e819ddf2297 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:52.083045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:52.083833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:52.084113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:52.084415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:52.095421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:52.096999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:52.114966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:52.116288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:52.117288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:52.117288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:52.117288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:52.117288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:52.117288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 614/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:34:52.123007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:52.227526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42688204-ba54-4dfe-a6f2-c46293920593 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:52.263262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:52.263934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:52.264244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:52.264516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:52.265254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:52.265548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:52.266944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:52.281183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 611/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:52.287561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:52.404535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460efc0e-d61e-47d9-bde7-f9920dda0d6c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:52.440079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:52.440939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:52.442676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:52.442676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:52.448764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:52.450528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:34:52.465380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:52.466530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:52.467925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 615/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:52.472804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:52.574211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077760d-aa7d-4b7e-8aeb-7847e8742958 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:52.611180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:52.611918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:52.612292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:52.612567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:52.613418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:52.613729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:52.615255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:52.641739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 612/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 12 17:34:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:52.642947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:52.748188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2662-191f-44a1-bc72-62d9f3290f9f None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:52.785187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:52.785975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:52.786253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:52.786528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:52.797767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:52.799345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:52.818099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:52.819004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:52.825085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:52.825085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:52.825085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:52.825085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:52.825085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 616/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 12 17:34:52.826314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:52.934739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4f30b-231d-402d-828f-79e2cc5ce33d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:52.972775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:52.973674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:52.974097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:52.974379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:52.975111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:52.975405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:52.976787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:52.988783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 613/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:52.994881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:53.109940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e6d6d-a54d-46a3-8f33-5bd32c9b598d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:53.146527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:53.147554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:53.147840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:53.148231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:53.156472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:53.158043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:34:53.174383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:53.175493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:53.176870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 617/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:34:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:53.181682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:53.288884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f75d7-7ee2-491b-a16e-3b879ecc5bc9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:53.324253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:53.324990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:53.325271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:53.325546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:53.326423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:53.326747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:53.328228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1fc6b-4c7d-4474-a7db-99b17b6728a1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:53.341404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 614/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:34:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:53.347332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:53.453067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f62d-15c2-4c81-ab59-27e35b7d803c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:53.487702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b2815-aad7-4fc5-87d0-768d68ac5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:53.488620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b2815-aad7-4fc5-87d0-768d68ac5784 None admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:53.488903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b2815-aad7-4fc5-87d0-768d68ac5784 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:53.489179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b2815-aad7-4fc5-87d0-768d68ac5784 None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/3704ba4cfa7f4ad8b798636b01ebd95b/roles/5c3b92ef9e6444d29a46a412d563184a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:53.505276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b2815-aad7-4fc5-87d0-768d68ac5784 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=3704ba4cfa7f4ad8b798636b01ebd95b, role_id=5c3b92ef9e6444d29a46a412d563184a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:53.506872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b2815-aad7-4fc5-87d0-768d68ac5784 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:53.519557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 618/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:34:53 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/3704ba4cfa7f4ad8b798636b01ebd95b/roles/5c3b92ef9e6444d29a46a412d563184a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:34:55.979602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136cd65-e0a2-4338-a0d1-e415642fc0c4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:55.980232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136cd65-e0a2-4338-a0d1-e415642fc0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:55.980232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136cd65-e0a2-4338-a0d1-e415642fc0c4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:55.981717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 615/1233] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:34:55.987723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:55.988209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:55.988318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:56.007206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:34:56.008432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:34:56.089942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 619/1234] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:34:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:34:56.124888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136cd65-e0a2-4338-a0d1-e415642fc0c4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:56.233278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136cd65-e0a2-4338-a0d1-e415642fc0c4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:56.267355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:56.268149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:56.268457 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:56.268731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:56.279143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:34:56.287359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:56.288596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:34:56.294836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 616/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Thu May 12 17:34:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:56.295698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:56.402352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298b638-2192-4f1d-93cc-870e1d4fc509 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:56.443074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:56.443806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:56.444095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:56.444433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:56.445246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] RBAC: Adding query filter params (name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:56.445573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:56.447036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:56.460884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 620/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Thu May 12 17:34:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:34:56.462161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:56.575352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf8546-e0e0-4fe6-a305-a3a6fa1ece50 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:56.611763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:56.612607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:56.612892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:56.613171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:56.625091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:56.626885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:56.650130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:56.651197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:56.652956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:56.652956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:56.652956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:56.652956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:34:56.652956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 617/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:56.659546 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:56.774360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b346d74-b1f2-429c-8cee-6a8ec42f8cdc None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:56.813965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:56.814628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:56.814911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:56.815187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:56.815912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:56.816249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:56.817616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:56.832129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 621/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:56.839382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:56.958824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fcd43-3e93-4c8d-9b6d-b6fd5fa2bcd5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:56.995479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:56.996846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:56.997115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:56.997453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:57.006209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:57.007698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:34:57.024448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:57.025626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:34:57.026639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 618/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:57.031787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:57.138765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dcd47-75b6-4fa3-bc93-8011eeef4782 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:57.175154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:57.175877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:57.176138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:57.176475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:57.177357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:57.177718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:57.179224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:57.197650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 622/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 12 17:34:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:57.203397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:57.313855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24a88c-44f2-43f1-9c46-bb2d393696dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:57.350965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:57.351678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:57.352008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:57.352347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:57.424737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:57.426352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:57.540743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 619/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:34:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:34:57.547598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:57.662752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb248-0f8d-4eb1-9fce-e0617c3088bf None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:57.699805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:57.700548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:57.700846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:57.701127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:57.711898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:57.713518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:34:57.733097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:57.739077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:57.740127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:34:57.740127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:34:57.740127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:34:57.740127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:57.740127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 623/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:57.740478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:57.847936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d3399-26f4-4219-9fe8-94d3ab91ff66 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:57.885214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:57.886002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:57.886780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:57.887201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:57.887998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:57.888390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:57.890056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:57.903456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 620/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:34:57.909052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:58.019221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee45b8-9908-4222-bc83-8028d6e2e284 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:58.055958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:58.056791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:58.057074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:58.057348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:58.065945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:58.067614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:34:58.083548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:34:58.084698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:34:58.085806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 624/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:34:58.090790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:58.202980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0015d-b118-4c26-904e-e9c21a7112c7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:58.238874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:58.239579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:58.239875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:58.240244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:58.241103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:58.241370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:58.243231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0830f-a8fa-46f4-96db-9d12fdde8ec9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:58.256172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 621/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:34:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:34:58.262896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:34:58.371407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c5cf3-4534-4797-976c-b833e31b2667 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:34:58.406699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:34:58.407417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:34:58.407752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:34:58.408120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:34:58.409035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] RBAC: Adding query filter params (role.id=5c3b92ef9e6444d29a46a412d563184a, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3704ba4cfa7f4ad8b798636b01ebd95b) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:34:58.409415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:34:58.411113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe40f98-21c4-4e42-b912-5a11c3ca7474 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:34:58.418497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 625/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:34:58 2022] GET /identity/v3/role_assignments?user.id=3704ba4cfa7f4ad8b798636b01ebd95b&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=5c3b92ef9e6444d29a46a412d563184a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:35:00.921722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd367-0618-4820-9e34-1c0a21ed47ee None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:00.922304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd367-0618-4820-9e34-1c0a21ed47ee None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:00.922304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd367-0618-4820-9e34-1c0a21ed47ee None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:00.923766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 622/1247] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:35:00.929339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:00.929807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:00.929928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:00.947605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:00.949426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:01.021380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 626/1248] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:35:01.049362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd367-0618-4820-9e34-1c0a21ed47ee None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:01.161314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd367-0618-4820-9e34-1c0a21ed47ee None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:01.196997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:01.197713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:01.197994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:01.198269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:01.209241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:35:01.217333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:01.218525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:01.219400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 623/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:01.225204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:01.328820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8111c3-6a23-487e-975f-22d9156cab1b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:01.364324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:01.364994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:01.365342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:01.365590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:01.366397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:01.366732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:01.368493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:01.376977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 627/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:35:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:35:01.382912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:01.493992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4234d5-8955-4293-bf5a-aa0c8941d3b7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:01.527686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:01.528730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:01.529078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:01.529406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:01.540676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:01.542112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:35:01.559740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:01.560731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:01.561633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:35:01.561633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:35:01.561633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:01.561633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:01.561633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 624/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:01.567067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:01.682394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f44ad-d4c5-4509-8528-f0b2cf6d30dd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:01.721140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:01.721875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:01.722159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:01.722436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:01.723239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:01.723589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:01.724999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:01.739194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 628/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:01.747821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:01.858757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9f43-c0ca-4dce-8c30-3aaaccf85265 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:01.901152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:01.902047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:01.902333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:01.902697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:01.911024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:01.912899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:35:01.929280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:01.931013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:01.932195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 625/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:01.936867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:02.039215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af86e64-e63a-4b39-a684-a17a9f30d2dd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:02.073468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:02.074167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:02.074556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:02.074826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:02.075651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:02.075958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:02.077408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:02.094909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 629/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 12 17:35:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:02.100661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:02.207339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60802ab4-82fb-476b-b736-db8ba03299c8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:02.241779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:02.242421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:02.242702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:02.243221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:02.312330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:02.313710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:02.417157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 626/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:35:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:35:02.425547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:02.535135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a70d5-214f-4802-ac23-10d2557327ca None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:02.571616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:02.572356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:02.572772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:02.573044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:02.585930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:02.587398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:35:02.607540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:02.608749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:02.609795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:35:02.609795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:35:02.609795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:02.609795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:02.609795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 630/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:02.614671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:02.722267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce32d5-552d-4517-8853-d71c2ee256ad None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:02.757552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:02.758224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:02.758500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:02.758767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:02.759493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:02.759794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:02.761209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:02.773254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 627/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:02.779498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:02.909842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89953135-8dd4-400a-a118-f6955d8d841c None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:02.944560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:02.945347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:02.945624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:02.945894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:02.953177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:02.954616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:35:02.968561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:02.969641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:02.970700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 631/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:35:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:02.975700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:03.081073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e348b5e-62b2-44cd-a6c0-10908cf08004 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:03.121314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:03.121607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:03.121885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:03.122152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:03.122935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:03.123240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:03.124688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cb042-78d1-4ce6-8c64-712c414b5f0c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:03.137465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 628/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:35:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:03.143525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:03.238916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1a44-ed40-4b17-825a-874f01322a2b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:03.269855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:03.270486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:03.270762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:03.271032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:03.271843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3704ba4cfa7f4ad8b798636b01ebd95b) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:03.272145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:03.273582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce10e61-6836-4ce7-b6ae-194765bae228 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:03.279423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 632/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:35:03 2022] GET /identity/v3/role_assignments?user.id=3704ba4cfa7f4ad8b798636b01ebd95b&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:05.527417 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da013b20-9711-4c86-ac2c-223ebad77fb8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:05.527952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da013b20-9711-4c86-ac2c-223ebad77fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:05.527952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da013b20-9711-4c86-ac2c-223ebad77fb8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:05.529327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 629/1261] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:35:05.534492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:05.534909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:05.535152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:05.551640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:05.552837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:05.615575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 633/1262] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 12 17:35:05.638671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da013b20-9711-4c86-ac2c-223ebad77fb8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:05.732633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da013b20-9711-4c86-ac2c-223ebad77fb8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:05.764348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:05.765121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:05.765402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:05.765670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:05.775120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:35:05.783410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:05.784557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:05.785576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 630/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:35:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:05.790365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:05.896769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a979f8-18c4-470d-acfa-ef8ab6109f87 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:05.931949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:05.932631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:05.932910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:05.933205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:05.934050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:05.934349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:05.935689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:05.943170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 634/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:35:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:05.949201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.040722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b2d66-6bba-4136-8b70-89fa51e29e82 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:06.071198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:06.071951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:06.072255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:06.072523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:06.082440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:06.083851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:35:06.100367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:06.101419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:06.102527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:35:06.102527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:35:06.102527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:06.102527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:06.102527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 631/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:06.107908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.205148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755f089-f67b-40a5-b39e-ede0ac20be3b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:06.235624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:06.236284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:06.236561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:06.236829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:06.237531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:06.237829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:06.239159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:06.250392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 635/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:06.255975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.348460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11c9a-1fa8-4c54-8664-e15a65fe77ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:06.379171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:06.379932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:06.380233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:06.380505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:06.388059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:06.389492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:35:06.403808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:06.404856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:06.405931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 632/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:06.410730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.506352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ec3ac-e8ac-4830-bb9f-e9dc887790fd None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:06.538839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:06.539497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:06.539780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:06.540058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:06.540927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:06.541237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:06.543650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:06.560812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 636/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 12 17:35:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:06.566535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.663288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97781b-22b4-43bf-b574-c8232dbd1076 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:06.695201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:06.695973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:06.696287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:06.696565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:06.706862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:06.708322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:35:06.730944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:06.732004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:06.733229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:35:06.733229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:35:06.733229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:06.733229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:06.733229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 633/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:06.739022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.844569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff472c-cba2-492a-80fd-be624619dc4b None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:06.885794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:06.886996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:06.887284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:06.887556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:06.888306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:06.888613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:06.889996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:06.900702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 637/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:06.906961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:06.998627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962c77b-e937-4faa-bc70-2c668a3683ff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:07.029589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:07.030344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:07.030619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:07.030883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:07.037093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:07.038503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:35:07.051906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:07.052973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:07.054090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 634/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:35:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:07.058686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:07.152989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7c2bf-6448-485a-9e0f-209e1c96e924 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:07.183818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:07.184486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:07.184773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:07.185048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:07.185822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:07.186130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:07.187541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fb72c-1ddc-43c0-993a-147a849d8244 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:07.199405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 638/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:35:07.205278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:07.298257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87283861-19e9-47e0-8cb7-26aba8c028ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:07.330001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc71b1-4fbf-4acc-909d-a2e24663cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:07.330861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc71b1-4fbf-4acc-909d-a2e24663cddf None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:07.331144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc71b1-4fbf-4acc-909d-a2e24663cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:07.331414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc71b1-4fbf-4acc-909d-a2e24663cddf None admin] PATH_INFO: `/v3/projects/ed2e282f526442d591d4fa647390748c/users/3704ba4cfa7f4ad8b798636b01ebd95b/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:07.333346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc71b1-4fbf-4acc-909d-a2e24663cddf None admin] RBAC: Authorizing `identity:create_grant(project_id=ed2e282f526442d591d4fa647390748c, user_id=3704ba4cfa7f4ad8b798636b01ebd95b, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:07.334754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc71b1-4fbf-4acc-909d-a2e24663cddf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:07.347196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 635/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Thu May 12 17:35:07 2022] PUT /identity/v3/projects/ed2e282f526442d591d4fa647390748c/users/3704ba4cfa7f4ad8b798636b01ebd95b/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:35:09.593353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2819a-730c-4a00-b09e-a0e26fb02fad None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:09.593881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2819a-730c-4a00-b09e-a0e26fb02fad None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:09.593995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2819a-730c-4a00-b09e-a0e26fb02fad None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:09.595256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 639/1274] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:35:09.600470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:09.600891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:09.601030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:09.617672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:09.619104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:09.687653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 636/1275] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:35:09.712006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2819a-730c-4a00-b09e-a0e26fb02fad None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:09.807923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2819a-730c-4a00-b09e-a0e26fb02fad None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:09.844332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:09.845342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:09.845717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:09.846050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:09.859230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(role_id) May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 12 17:35:09.867223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:09.868480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:09.869341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 640/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Thu May 12 17:35:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:09.875599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:09.977385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117988c-f7f8-403c-829b-681d7a584a87 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:10.009230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:10.009881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:10.010158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:10.010427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:10.011186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:10.011489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:10.012885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:10.020435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 637/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Thu May 12 17:35:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:10.026110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:10.119148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f724db3-4b69-4693-ada6-3e934c746fc3 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:10.150807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:10.151582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:10.152308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:10.152422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:10.162359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:10.163800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:35:10.180514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:10.181590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:10.182733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:35:10.182733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:35:10.182733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:10.182733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:10.182733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 641/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:10.188341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:10.281763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31e30a-cece-4d8e-87a7-c872390f2e0c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:10.312406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:10.313047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:10.313327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:10.313601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:10.314306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:10.314606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:10.315948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:10.327074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 638/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:10.332604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:10.427840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837daf2-0c4a-42c2-8a9b-beeb35cfc3cb None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:10.458342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:10.459111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:10.459393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:10.459665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:10.467321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:10.468782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_user(user_id) May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 12 17:35:10.482861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:10.484025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:10.485235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 642/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:10.489707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:10.581106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d305b11-fe87-4ed1-8cf3-93b42f5e89fd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:10.613548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:10.614210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:10.614489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:10.614761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:10.615628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:10.615870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:10.617304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:10.633726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 639/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Thu May 12 17:35:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:10.639938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:10.734101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50d22b-4e28-4923-baa7-5ff638d86fa8 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:10.766476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:10.767111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:10.767395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:10.767672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:10.829632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:10.831233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:10.928677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 643/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Thu May 12 17:35:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 12 17:35:10.935141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:11.027900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab89d9-52fe-41af-be4b-0074f0151cbc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:11.059106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:11.059866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:11.060145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:11.060448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:11.069747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:11.071166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 12 17:35:11.087617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:11.088682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:11.094045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 12 17:35:11.094045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 12 17:35:11.094045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 12 17:35:11.094045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:11.094045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 640/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:11.095049 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:11.188026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1843d7f-2d48-4ece-8049-7b35cab3ffab None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:11.219403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:11.220047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:11.220359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:11.220636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:11.221347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:11.221652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:11.222996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:11.234156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 644/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Thu May 12 17:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:11.239715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:11.330361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6220895-ddda-48f2-a5d8-97064200917f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:11.361848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:11.362611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:11.362885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:11.363152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:11.369703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:11.371136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) May 12 17:35:11.384107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:11.385113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application May 12 17:35:11.386018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 641/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Thu May 12 17:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:11.391542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:11.488265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf8183-d788-4291-8f99-20872656db3e None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:11.520098 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:11.520774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:11.521059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:11.521335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:11.522116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:11.522425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:11.523841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950b487-f3b3-4ec9-b2bd-6c67f0b5f9e1 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:11.535820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 645/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Thu May 12 17:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:11.541832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:11.634674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8ef2-e5a5-4afb-a189-0dc1268dc030 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:11.666435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:11.667066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:11.667367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:11.667642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:11.668485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] RBAC: Adding query filter params (role.id=9007f8fb775a4cc3875fe578b35f1c1d, scope.project.id=ed2e282f526442d591d4fa647390748c, user.id=3704ba4cfa7f4ad8b798636b01ebd95b) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:11.668791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:11.670202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0b0ba-8a90-49b8-8361-dc66aa54c535 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:11.675683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 642/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Thu May 12 17:35:11 2022] GET /identity/v3/role_assignments?user.id=3704ba4cfa7f4ad8b798636b01ebd95b&scope.project.id=ed2e282f526442d591d4fa647390748c&role.id=9007f8fb775a4cc3875fe578b35f1c1d => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:13.910276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41470c50-5a13-442c-b243-3323283c27c0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:13.910803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41470c50-5a13-442c-b243-3323283c27c0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:13.910898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41470c50-5a13-442c-b243-3323283c27c0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:13.912219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 646/1288] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:35:13.917976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:13.918403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:13.918616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:13.936057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:13.937254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:14.000815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 643/1289] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:35:14.022774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41470c50-5a13-442c-b243-3323283c27c0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:14.117731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41470c50-5a13-442c-b243-3323283c27c0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:14.148391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:14.149172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:14.149451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:14.149749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:14.150387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:14.152343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:14.163658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:14.164884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:14.165927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 647/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:35:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:14.171966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:14.268526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe79d7-18ea-455c-85d1-9f743d33632d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:14.301354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:14.302003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:14.302282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:14.302556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:14.303255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:14.303559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:14.304940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1596877-df3a-446d-b8dd-6fa9a7406796 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:14.309984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 644/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 12 17:35:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:14.315511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:14.409055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5e8f7-ce97-45bc-a11d-7036a3e94f83 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:14.440222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:14.440817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:14.441100 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:14.441378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] PATH_INFO: `/v3/services` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:14.442079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:14.442384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:14.443738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ec461-e3fe-4be9-9c98-25215a7ac5f3 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:14.447981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 648/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 12 17:35:14 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:35:16.646185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf772eef-1e74-4f0a-87ab-4db78dd3db62 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:16.646644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf772eef-1e74-4f0a-87ab-4db78dd3db62 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:16.646733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf772eef-1e74-4f0a-87ab-4db78dd3db62 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:16.648080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 645/1293] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:35:16.653317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabf68-a2d0-493c-a081-2f996160e496 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:16.653835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabf68-a2d0-493c-a081-2f996160e496 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:16.653835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabf68-a2d0-493c-a081-2f996160e496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:16.670309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7bfabf68-a2d0-493c-a081-2f996160e496 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:16.671470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7bfabf68-a2d0-493c-a081-2f996160e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:16.732415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 649/1294] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:35:16.755120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf772eef-1e74-4f0a-87ab-4db78dd3db62 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:16.854684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf772eef-1e74-4f0a-87ab-4db78dd3db62 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:16.889051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b135ce-ce1e-47ad-b9af-c9d0068abcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:16.889580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b135ce-ce1e-47ad-b9af-c9d0068abcaa None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:16.889859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b135ce-ce1e-47ad-b9af-c9d0068abcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:16.890138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b135ce-ce1e-47ad-b9af-c9d0068abcaa None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:16.890922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b135ce-ce1e-47ad-b9af-c9d0068abcaa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:16.892488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b135ce-ce1e-47ad-b9af-c9d0068abcaa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:16.908108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 646/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Thu May 12 17:35:16 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:35:19.141603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b8b92-8174-4629-9aac-2477faedeb0c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:19.142141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b8b92-8174-4629-9aac-2477faedeb0c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:19.142238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b8b92-8174-4629-9aac-2477faedeb0c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:19.143547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 650/1296] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:35:19.149123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:19.149583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:19.149679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:19.182639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:19.183912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:19.264918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 647/1297] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:35:19.288310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b8b92-8174-4629-9aac-2477faedeb0c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:19.386851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b8b92-8174-4629-9aac-2477faedeb0c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:19.419483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:19.420331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:19.420616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:19.420890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:19.421536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:19.422896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:19.436043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:19.437269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:19.438536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 651/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:35:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:19.443621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:19.541993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d91f4-1594-4852-a5b0-12f683e007b1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:19.573167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:19.574459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:19.574738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:19.575008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:19.575707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:19.576009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:19.577396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:19.582256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 648/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 12 17:35:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:19.588238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:19.682720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1196c-4a5b-4ed5-bb18-26b6dd73372d None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:19.715807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:19.716478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:19.716763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:19.717039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:19.717821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] RBAC: Adding query filter params (interface=public, service_id=9170df3bafe84f809abec90b7baa1aeb, region_id=RegionOne) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:19.718130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:19.719475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1a26a-c704-4076-b3e8-61bf492d677f None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:19.724271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 652/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Thu May 12 17:35:19 2022] GET /identity/v3/endpoints?service_id=9170df3bafe84f809abec90b7baa1aeb&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:19.730533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:19.826408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfeee4-ec82-4852-aa02-7be529146b2f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:19.869127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052aced-99e0-46ab-aba0-8535cb780ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:19.869781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052aced-99e0-46ab-aba0-8535cb780ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:19.870058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052aced-99e0-46ab-aba0-8535cb780ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:19.870343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052aced-99e0-46ab-aba0-8535cb780ec9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:19.871615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052aced-99e0-46ab-aba0-8535cb780ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:19.873002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052aced-99e0-46ab-aba0-8535cb780ec9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:19.879637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 649/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Thu May 12 17:35:19 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:35:22.085071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7043be-40c4-4127-b7d2-1604e15e733b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:22.085593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7043be-40c4-4127-b7d2-1604e15e733b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:22.085593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7043be-40c4-4127-b7d2-1604e15e733b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:22.086945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 653/1302] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:35:22.092444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:22.092823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:22.093005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:22.110085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:35:22.111244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:35:22.172750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 650/1303] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:35:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 12 17:35:22.196079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7043be-40c4-4127-b7d2-1604e15e733b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:22.290352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7043be-40c4-4127-b7d2-1604e15e733b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:22.322550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:22.323333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:22.323647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:22.323916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:22.324590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:22.325931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application Traceback (most recent call last): May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 12 17:35:22.337376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._get_service(service_id) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 12 17:35:22.338514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.get_or_create( May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application with Lock( May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self._enter() May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application return self.creator() May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application created_value = creator( May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: ERROR keystone.server.flask.application May 12 17:35:22.340352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 654/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Thu May 12 17:35:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 12 17:35:22.345136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:22.436956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243b4f-178a-4dc4-9d7c-bdade41c8be1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:22.468556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:22.469201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:22.469481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:22.469754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:22.470455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:35:22.470760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:22.472124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8327b-f722-4b9e-a63f-c04c1351c6a5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:22.476993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 651/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Thu May 12 17:35:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:35:22.483187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:35:22.575870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ce427-ae33-4604-81be-6160c3c2d481 None admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:35:22.607777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8390f-0931-4f72-a1dd-65d1307a7824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:35:22.608383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8390f-0931-4f72-a1dd-65d1307a7824 None admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:35:22.608663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8390f-0931-4f72-a1dd-65d1307a7824 None admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:35:22.608932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8390f-0931-4f72-a1dd-65d1307a7824 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:35:22.609650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8390f-0931-4f72-a1dd-65d1307a7824 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:35:22.611015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8390f-0931-4f72-a1dd-65d1307a7824 None admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:35:22.628506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 655/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Thu May 12 17:35:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:36:26.500255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d451243-6378-4cff-9010-d8079418fbcb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:26.501319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d451243-6378-4cff-9010-d8079418fbcb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:26.501821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d451243-6378-4cff-9010-d8079418fbcb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:26.503687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 652/1307] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:26.508792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fed8e3-771c-45c6-a6b6-482d9b5816c7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:26.509593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fed8e3-771c-45c6-a6b6-482d9b5816c7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:26.510019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fed8e3-771c-45c6-a6b6-482d9b5816c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:26.598229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-83fed8e3-771c-45c6-a6b6-482d9b5816c7 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:26.600875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-83fed8e3-771c-45c6-a6b6-482d9b5816c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:26.750102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 656/1308] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:26.814679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbae6c-5781-441c-b354-d375add48856 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:26.814953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbae6c-5781-441c-b354-d375add48856 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:26.815210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbae6c-5781-441c-b354-d375add48856 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:26.817059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 653/1309] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 12 17:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:26.853789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44f13c-5296-4d16-9cac-768500283c16 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:26.854105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44f13c-5296-4d16-9cac-768500283c16 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:26.854386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44f13c-5296-4d16-9cac-768500283c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:26.986017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6c44f13c-5296-4d16-9cac-768500283c16 None None] MFA Rules not processed for user `6076a5f71eed4f8fa47d74a79d6fa709`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:26.987373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6c44f13c-5296-4d16-9cac-768500283c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:27.084259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 657/1310] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 12 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:27.085068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbae6c-5781-441c-b354-d375add48856 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:27.087035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-64bbae6c-5781-441c-b354-d375add48856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:27.295642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbae6c-5781-441c-b354-d375add48856 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:27.391347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:27.392331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:27.392788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:27.393277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:27.396872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:27.515593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:27.518328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264f199-01c6-4774-9bf9-49382da092f2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:27.541820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 654/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:36:30.122384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f5b4-febc-4749-b173-3d3842246903 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:30.122944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f5b4-febc-4749-b173-3d3842246903 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:30.122944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f5b4-febc-4749-b173-3d3842246903 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:30.124334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 658/1312] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:30.129809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19fdf5-e400-4ed9-8e5d-3210b1d80ede None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:30.130262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19fdf5-e400-4ed9-8e5d-3210b1d80ede None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:30.130371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19fdf5-e400-4ed9-8e5d-3210b1d80ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:30.214978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1b19fdf5-e400-4ed9-8e5d-3210b1d80ede None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:30.216239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1b19fdf5-e400-4ed9-8e5d-3210b1d80ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:30.323659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 655/1313] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:30.361469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699f5b4-febc-4749-b173-3d3842246903 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:30.363009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f699f5b4-febc-4749-b173-3d3842246903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:30.464137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699f5b4-febc-4749-b173-3d3842246903 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:30.534295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:30.534946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:30.535230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:30.535534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:30.537203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:30.649126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:30.650567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c47b8-ffac-4e24-b7cc-1d4409527c7e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:30.669908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 659/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:33.198491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4986a-3ade-45a8-9a64-c312d3e325e1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:33.199643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4986a-3ade-45a8-9a64-c312d3e325e1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:33.199643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4986a-3ade-45a8-9a64-c312d3e325e1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:33.200593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 656/1315] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:36:33.205745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148b67-a7a2-4830-a42b-59bbdd6499d2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:33.206142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148b67-a7a2-4830-a42b-59bbdd6499d2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:33.206304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148b67-a7a2-4830-a42b-59bbdd6499d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:33.267649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5c148b67-a7a2-4830-a42b-59bbdd6499d2 None None] MFA Rules not processed for user `dbe1e8845ac542ff9fb11a8bbaa70f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:33.268879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5c148b67-a7a2-4830-a42b-59bbdd6499d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:33.359448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 660/1316] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:33.397891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4986a-3ade-45a8-9a64-c312d3e325e1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:33.399398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2dd4986a-3ade-45a8-9a64-c312d3e325e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:33.497275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4986a-3ade-45a8-9a64-c312d3e325e1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:33.565221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:33.565862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:33.566141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:33.566414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:33.568017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:33.670954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:33.672434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6613ce-e56a-4e3b-9f43-f8dd9fb040ab service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:33.689881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 657/1317] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:36.441136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe716c-7bf2-409f-a59b-3d90bf86e7ca None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:36.441710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe716c-7bf2-409f-a59b-3d90bf86e7ca None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:36.441710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe716c-7bf2-409f-a59b-3d90bf86e7ca None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:36.443044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 661/1318] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:36.448331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d931-a558-4991-8e2f-04e137d05afa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:36.448749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d931-a558-4991-8e2f-04e137d05afa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:36.448894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d931-a558-4991-8e2f-04e137d05afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:36.514427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cfc4d931-a558-4991-8e2f-04e137d05afa None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:36.515638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cfc4d931-a558-4991-8e2f-04e137d05afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:36.608079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 658/1319] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:36.665697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b75808-c7ec-45cd-89df-57cab05756fd None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:36.665982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b75808-c7ec-45cd-89df-57cab05756fd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:36.666258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b75808-c7ec-45cd-89df-57cab05756fd None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:36.667625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 662/1320] 10.222.0.73 () {54 vars in 964 bytes} [Thu May 12 17:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:36.684065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8de445-6283-4992-a77b-91ad34eec128 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:36.684384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8de445-6283-4992-a77b-91ad34eec128 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:36.684688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8de445-6283-4992-a77b-91ad34eec128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:36.756445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3d8de445-6283-4992-a77b-91ad34eec128 None None] MFA Rules not processed for user `6076a5f71eed4f8fa47d74a79d6fa709`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:36.757547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3d8de445-6283-4992-a77b-91ad34eec128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:36.829285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 659/1321] 10.222.0.73 () {58 vars in 1061 bytes} [Thu May 12 17:36:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:36.838342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b75808-c7ec-45cd-89df-57cab05756fd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:36.840788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-65b75808-c7ec-45cd-89df-57cab05756fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:36.939183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b75808-c7ec-45cd-89df-57cab05756fd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:37.007442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:37.008008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:37.008317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:37.008590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:37.010199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:37.112716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:37.114140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee87e14-3f84-4707-9cd2-7e25b5c7d6a1 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:37.132139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 663/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:40.443376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7c85f-8009-464e-aa81-05320d04e54f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:40.443945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7c85f-8009-464e-aa81-05320d04e54f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:40.443945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7c85f-8009-464e-aa81-05320d04e54f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:40.445316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 660/1323] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:40.451072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56645-a7db-404c-b182-67baed73e1fd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:40.451516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56645-a7db-404c-b182-67baed73e1fd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:40.451619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56645-a7db-404c-b182-67baed73e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:40.514407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-10f56645-a7db-404c-b182-67baed73e1fd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:40.515583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-10f56645-a7db-404c-b182-67baed73e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:40.610675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 664/1324] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:40.647194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7c85f-8009-464e-aa81-05320d04e54f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:40.648763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fac7c85f-8009-464e-aa81-05320d04e54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:40.751656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7c85f-8009-464e-aa81-05320d04e54f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:40.822403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:40.823051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:40.823331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:40.823604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:40.826464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:40.934078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:40.935680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b2b85-eb09-4034-9a39-41f5c44cbaf1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:40.954880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 661/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:43.234607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b124df4-1777-4e6c-a026-c27699bd9508 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:43.235177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b124df4-1777-4e6c-a026-c27699bd9508 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:43.235177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b124df4-1777-4e6c-a026-c27699bd9508 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:43.236645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 665/1326] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:43.242677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf26c2-3f9d-4947-ac30-5daf5a4c9290 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:43.243271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf26c2-3f9d-4947-ac30-5daf5a4c9290 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:43.243271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf26c2-3f9d-4947-ac30-5daf5a4c9290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:43.307495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b4bf26c2-3f9d-4947-ac30-5daf5a4c9290 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:43.309350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4bf26c2-3f9d-4947-ac30-5daf5a4c9290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:43.402776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 662/1327] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:43.438804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b124df4-1777-4e6c-a026-c27699bd9508 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:43.440431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9b124df4-1777-4e6c-a026-c27699bd9508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:43.540937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b124df4-1777-4e6c-a026-c27699bd9508 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:43.611219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:43.611868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:43.612153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:43.612454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:43.614096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:43.718593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:43.720017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb97037-16a4-482c-a7cd-491d3007e0c0 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:43.738347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 666/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:46.076602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46abc59-6c90-428c-8311-c23ed9d590a4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:46.077138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46abc59-6c90-428c-8311-c23ed9d590a4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:46.077138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46abc59-6c90-428c-8311-c23ed9d590a4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:46.078548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 663/1329] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:46.083852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5a22f-c518-43c3-9d75-c9f7d492f380 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:46.084304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5a22f-c518-43c3-9d75-c9f7d492f380 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:46.084575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5a22f-c518-43c3-9d75-c9f7d492f380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:46.149779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-09d5a22f-c518-43c3-9d75-c9f7d492f380 None None] MFA Rules not processed for user `dbe1e8845ac542ff9fb11a8bbaa70f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:46.150981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-09d5a22f-c518-43c3-9d75-c9f7d492f380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:46.248599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 667/1330] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:46.285944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46abc59-6c90-428c-8311-c23ed9d590a4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:46.287561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f46abc59-6c90-428c-8311-c23ed9d590a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:46.389442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46abc59-6c90-428c-8311-c23ed9d590a4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:46.459308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:46.459953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:46.460257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:46.460528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:46.462130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:46.566280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:46.567697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef1bbc-8449-4f64-88a8-f978015537e2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:46.585927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 664/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:49.080244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356aac-6636-4ca5-9a13-8102a0c1852f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:49.080813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356aac-6636-4ca5-9a13-8102a0c1852f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:49.080813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356aac-6636-4ca5-9a13-8102a0c1852f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:49.082310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 668/1332] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:49.087495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106765c-30df-40c0-b1c2-b361d25c8f08 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:49.087960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106765c-30df-40c0-b1c2-b361d25c8f08 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:49.088052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106765c-30df-40c0-b1c2-b361d25c8f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:49.155362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0106765c-30df-40c0-b1c2-b361d25c8f08 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:49.156573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0106765c-30df-40c0-b1c2-b361d25c8f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:49.248908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 665/1333] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:49.283711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356aac-6636-4ca5-9a13-8102a0c1852f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:49.285283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-41356aac-6636-4ca5-9a13-8102a0c1852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:49.405867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356aac-6636-4ca5-9a13-8102a0c1852f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:49.475734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:49.476401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:49.476679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:49.476950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:49.478560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:49.581163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:49.582518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc33e4-09b2-47e6-890d-2744c24286ca service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:49.600187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 669/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:51.903349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d9e7e-4cdb-4360-aa08-28ceac94a02f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:51.903869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d9e7e-4cdb-4360-aa08-28ceac94a02f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:51.903960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d9e7e-4cdb-4360-aa08-28ceac94a02f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:51.905301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 666/1335] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:51.910565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254326e5-60a2-4f8c-9f09-4f802f433e54 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:51.910941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254326e5-60a2-4f8c-9f09-4f802f433e54 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:51.911224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254326e5-60a2-4f8c-9f09-4f802f433e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:51.974791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-254326e5-60a2-4f8c-9f09-4f802f433e54 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:51.975958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-254326e5-60a2-4f8c-9f09-4f802f433e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:52.072649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 670/1336] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:52.112854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d9e7e-4cdb-4360-aa08-28ceac94a02f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:52.114354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-586d9e7e-4cdb-4360-aa08-28ceac94a02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:52.212648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d9e7e-4cdb-4360-aa08-28ceac94a02f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:52.281629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:52.282269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:52.282550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:52.282822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:52.284455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:52.387581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:52.389035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212c4cf-1fdf-4a9d-8e6d-25cc7db29220 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:52.405937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 667/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:54.915929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92153bf4-f011-4e2b-8326-ccfc8b20ece6 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:54.916492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92153bf4-f011-4e2b-8326-ccfc8b20ece6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:54.916598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92153bf4-f011-4e2b-8326-ccfc8b20ece6 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:54.918657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 671/1338] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:54.923808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1dfb6-bc1a-41b0-a16c-c676605dfe76 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:54.924264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1dfb6-bc1a-41b0-a16c-c676605dfe76 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:54.924402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1dfb6-bc1a-41b0-a16c-c676605dfe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:54.987522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7ba1dfb6-bc1a-41b0-a16c-c676605dfe76 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:54.988746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7ba1dfb6-bc1a-41b0-a16c-c676605dfe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:55.081885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 668/1339] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:55.118282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92153bf4-f011-4e2b-8326-ccfc8b20ece6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:55.120063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-92153bf4-f011-4e2b-8326-ccfc8b20ece6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:55.218371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92153bf4-f011-4e2b-8326-ccfc8b20ece6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:36:55.287148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:36:55.287782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:55.288062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:55.288362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:55.289966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:55.402129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:36:55.403553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccec4bf-6cef-4d09-8d06-8d2fba30feba service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:36:55.420604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 672/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:36:59.737722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7892d-bdb7-4337-b4d1-140f60bbee49 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:59.738262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7892d-bdb7-4337-b4d1-140f60bbee49 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:59.738262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7892d-bdb7-4337-b4d1-140f60bbee49 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:59.739651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 669/1341] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:36:59.745364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced38c1c-231a-4bc8-a9b8-abec0ff0e5db None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:36:59.745836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced38c1c-231a-4bc8-a9b8-abec0ff0e5db None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:36:59.745952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced38c1c-231a-4bc8-a9b8-abec0ff0e5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:36:59.817314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ced38c1c-231a-4bc8-a9b8-abec0ff0e5db None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:36:59.818445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ced38c1c-231a-4bc8-a9b8-abec0ff0e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:36:59.923206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 673/1342] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:36:59.961196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7892d-bdb7-4337-b4d1-140f60bbee49 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:36:59.963282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dbb7892d-bdb7-4337-b4d1-140f60bbee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:00.054185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7892d-bdb7-4337-b4d1-140f60bbee49 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:00.057701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f8740-25e7-4052-9cd4-350ae3546796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:00.058364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f8740-25e7-4052-9cd4-350ae3546796 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:00.058643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f8740-25e7-4052-9cd4-350ae3546796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:00.058910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f8740-25e7-4052-9cd4-350ae3546796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:00.067953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f8740-25e7-4052-9cd4-350ae3546796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:00.069679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f8740-25e7-4052-9cd4-350ae3546796 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:00.079002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 670/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:37:02.936625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378734-50ce-4c34-b089-a9ccb3c1f16b None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:02.937213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378734-50ce-4c34-b089-a9ccb3c1f16b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:02.937213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378734-50ce-4c34-b089-a9ccb3c1f16b None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:02.938543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 674/1344] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:02.944008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff12bd-5100-4158-b76f-f1f873f1fb66 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:02.944519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff12bd-5100-4158-b76f-f1f873f1fb66 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:02.944717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff12bd-5100-4158-b76f-f1f873f1fb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:02.972745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e3ff12bd-5100-4158-b76f-f1f873f1fb66 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:02.973845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e3ff12bd-5100-4158-b76f-f1f873f1fb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:03.046210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 671/1345] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:03.083615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8378734-50ce-4c34-b089-a9ccb3c1f16b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:03.093995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8378734-50ce-4c34-b089-a9ccb3c1f16b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:03.097673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88dfa-ead9-4d7e-902b-f88507e20f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:03.098401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88dfa-ead9-4d7e-902b-f88507e20f3d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:03.098747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88dfa-ead9-4d7e-902b-f88507e20f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:03.099085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88dfa-ead9-4d7e-902b-f88507e20f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:03.107959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d88dfa-ead9-4d7e-902b-f88507e20f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:03.109469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d88dfa-ead9-4d7e-902b-f88507e20f3d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:03.118759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 675/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:07.543902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b9ad7-d507-4232-8ca7-55061c4eb3e0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:07.544775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b9ad7-d507-4232-8ca7-55061c4eb3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:07.545059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b9ad7-d507-4232-8ca7-55061c4eb3e0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:07.546473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 672/1347] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:07.552191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a415c-04c9-45c4-922f-06119ad5c2c9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:07.552643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a415c-04c9-45c4-922f-06119ad5c2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:07.552746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a415c-04c9-45c4-922f-06119ad5c2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:07.569885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b00a415c-04c9-45c4-922f-06119ad5c2c9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:07.571047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b00a415c-04c9-45c4-922f-06119ad5c2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:07.579042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 676/1348] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:37:07.619065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b9ad7-d507-4232-8ca7-55061c4eb3e0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:07.630130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b9ad7-d507-4232-8ca7-55061c4eb3e0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:07.633530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759370b8-9c95-427a-8b66-96994c9ebe75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:07.634156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759370b8-9c95-427a-8b66-96994c9ebe75 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:07.634433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759370b8-9c95-427a-8b66-96994c9ebe75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:07.634699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759370b8-9c95-427a-8b66-96994c9ebe75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:07.643728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759370b8-9c95-427a-8b66-96994c9ebe75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:07.645180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759370b8-9c95-427a-8b66-96994c9ebe75 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:07.654939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 673/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:13.008058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a549a-3188-4811-a72f-36ed5ffc18b1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:13.009038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a549a-3188-4811-a72f-36ed5ffc18b1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:13.009553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a549a-3188-4811-a72f-36ed5ffc18b1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:13.012275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 677/1350] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:37:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:13.017930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06583db7-b62e-4579-99d2-37195c0aec07 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:13.018363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06583db7-b62e-4579-99d2-37195c0aec07 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:13.018473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06583db7-b62e-4579-99d2-37195c0aec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:13.177157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06583db7-b62e-4579-99d2-37195c0aec07 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:13.178370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06583db7-b62e-4579-99d2-37195c0aec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:13.296523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 674/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:37:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:17.523091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bd36b-32e9-44b7-b011-2b981e508583 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:17.523639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bd36b-32e9-44b7-b011-2b981e508583 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:17.524101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bd36b-32e9-44b7-b011-2b981e508583 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:17.525593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 678/1352] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:17.531535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489888cd-0bdb-4e76-971e-039def8d98f3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:17.531962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489888cd-0bdb-4e76-971e-039def8d98f3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:17.532192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489888cd-0bdb-4e76-971e-039def8d98f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:17.551392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-489888cd-0bdb-4e76-971e-039def8d98f3 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:17.552940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-489888cd-0bdb-4e76-971e-039def8d98f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:17.562583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 675/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:37:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:37:19.211662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0ab70-6316-4a67-a0fb-22ed7e95545d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:19.212670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0ab70-6316-4a67-a0fb-22ed7e95545d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:19.212670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0ab70-6316-4a67-a0fb-22ed7e95545d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:19.213665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 679/1354] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:19.220600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04124839-6fde-4d72-9f76-93d8e08c5c23 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:19.221064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04124839-6fde-4d72-9f76-93d8e08c5c23 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:19.221156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04124839-6fde-4d72-9f76-93d8e08c5c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:19.239581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-04124839-6fde-4d72-9f76-93d8e08c5c23 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:19.240796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04124839-6fde-4d72-9f76-93d8e08c5c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:19.249708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 676/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:37:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:23.220615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366dda-d4a3-41bc-8035-f0665c4ba7cd None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:23.221186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366dda-d4a3-41bc-8035-f0665c4ba7cd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:23.221186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366dda-d4a3-41bc-8035-f0665c4ba7cd None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:23.222509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 680/1356] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:23.227806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23df48a-5306-4f5a-8c6e-1aec8bb24531 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:23.228237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23df48a-5306-4f5a-8c6e-1aec8bb24531 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:23.228414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23df48a-5306-4f5a-8c6e-1aec8bb24531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:23.247390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f23df48a-5306-4f5a-8c6e-1aec8bb24531 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:23.248622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f23df48a-5306-4f5a-8c6e-1aec8bb24531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:23.256661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 677/1357] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:23.305073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aace4c-0416-4c84-b28f-15d8fa37808f None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:23.305352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aace4c-0416-4c84-b28f-15d8fa37808f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:23.305623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aace4c-0416-4c84-b28f-15d8fa37808f None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:23.307108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 681/1358] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 12 17:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:37:23.313132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41dfe3-fc34-4b05-8878-8e5d3f95bc23 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:23.313425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41dfe3-fc34-4b05-8878-8e5d3f95bc23 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:23.313706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41dfe3-fc34-4b05-8878-8e5d3f95bc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:23.362195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fb41dfe3-fc34-4b05-8878-8e5d3f95bc23 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:23.363496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fb41dfe3-fc34-4b05-8878-8e5d3f95bc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:23.444005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 678/1359] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 12 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:23.452129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aace4c-0416-4c84-b28f-15d8fa37808f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:23.467362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aace4c-0416-4c84-b28f-15d8fa37808f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:23.472937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939172a-324c-49ce-909e-60af26578919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:23.473612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939172a-324c-49ce-909e-60af26578919 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:23.473989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939172a-324c-49ce-909e-60af26578919 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:23.474358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939172a-324c-49ce-909e-60af26578919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:23.483085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939172a-324c-49ce-909e-60af26578919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:23.484811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939172a-324c-49ce-909e-60af26578919 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:23.494421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 682/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:25.957113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f16cb5-ffb9-414e-bcbc-8108c41cbbff None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:25.957693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f16cb5-ffb9-414e-bcbc-8108c41cbbff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:25.957693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f16cb5-ffb9-414e-bcbc-8108c41cbbff None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:25.959022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 679/1361] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:25.965249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23a6d7-62c2-4455-a6d6-031c8f006e60 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:25.965698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23a6d7-62c2-4455-a6d6-031c8f006e60 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:25.965867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23a6d7-62c2-4455-a6d6-031c8f006e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:25.983725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5e23a6d7-62c2-4455-a6d6-031c8f006e60 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:25.985068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5e23a6d7-62c2-4455-a6d6-031c8f006e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:25.993236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 683/1362] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:26.038206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f16cb5-ffb9-414e-bcbc-8108c41cbbff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:26.052774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f16cb5-ffb9-414e-bcbc-8108c41cbbff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:26.059052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafecaee-0681-42c7-a7a5-031afe4a3297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:26.059861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafecaee-0681-42c7-a7a5-031afe4a3297 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:26.060142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafecaee-0681-42c7-a7a5-031afe4a3297 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:26.060446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafecaee-0681-42c7-a7a5-031afe4a3297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:26.068854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafecaee-0681-42c7-a7a5-031afe4a3297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:26.070475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafecaee-0681-42c7-a7a5-031afe4a3297 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:26.081068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 680/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:28.633358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e4ab-b36a-445f-a6ff-492c56a6a97c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:28.633819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e4ab-b36a-445f-a6ff-492c56a6a97c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:28.633915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e4ab-b36a-445f-a6ff-492c56a6a97c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:28.635577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 684/1364] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:28.642229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b120-a54e-4140-bd96-d4a23a44d2d5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:28.642704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b120-a54e-4140-bd96-d4a23a44d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:28.642799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b120-a54e-4140-bd96-d4a23a44d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:28.660045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22b1b120-a54e-4140-bd96-d4a23a44d2d5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:28.661251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22b1b120-a54e-4140-bd96-d4a23a44d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:28.673065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 681/1365] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:28.718493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eac1e6-11af-4a23-8bac-13d71427e043 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:28.718892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eac1e6-11af-4a23-8bac-13d71427e043 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:28.719385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eac1e6-11af-4a23-8bac-13d71427e043 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:28.721189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 685/1366] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 12 17:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:28.728043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db639-9440-4f3e-ac4b-d97312727df5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:28.728632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db639-9440-4f3e-ac4b-d97312727df5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:28.729101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db639-9440-4f3e-ac4b-d97312727df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:28.759654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-760db639-9440-4f3e-ac4b-d97312727df5 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:28.761249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-760db639-9440-4f3e-ac4b-d97312727df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:28.770829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 682/1367] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 12 17:37:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:28.778912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eac1e6-11af-4a23-8bac-13d71427e043 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:28.793410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eac1e6-11af-4a23-8bac-13d71427e043 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:28.798812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8c33c-78c7-4cdd-98e6-6bb72a7eaac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:28.799509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8c33c-78c7-4cdd-98e6-6bb72a7eaac2 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:28.799849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8c33c-78c7-4cdd-98e6-6bb72a7eaac2 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:28.800242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8c33c-78c7-4cdd-98e6-6bb72a7eaac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:28.808896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8c33c-78c7-4cdd-98e6-6bb72a7eaac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:28.810457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8c33c-78c7-4cdd-98e6-6bb72a7eaac2 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:28.819461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 686/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:31.141402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0b33-0f91-47b9-bb51-f72234a3c40a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.141949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0b33-0f91-47b9-bb51-f72234a3c40a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.141949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a0b33-0f91-47b9-bb51-f72234a3c40a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.143218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 683/1369] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:31.148859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f03d-6f26-4cad-a811-63de1604e8e0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.149295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f03d-6f26-4cad-a811-63de1604e8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.149497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f03d-6f26-4cad-a811-63de1604e8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.168202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cc94f03d-6f26-4cad-a811-63de1604e8e0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:31.169672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cc94f03d-6f26-4cad-a811-63de1604e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:31.184051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 687/1370] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:31.228572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0b33-0f91-47b9-bb51-f72234a3c40a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:31.242934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a0b33-0f91-47b9-bb51-f72234a3c40a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:31.248919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f9407-ed3a-4c5e-9368-6f2e6f5702bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:31.249681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f9407-ed3a-4c5e-9368-6f2e6f5702bb service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.249980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f9407-ed3a-4c5e-9368-6f2e6f5702bb service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.250245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f9407-ed3a-4c5e-9368-6f2e6f5702bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.258497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f9407-ed3a-4c5e-9368-6f2e6f5702bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:31.260214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f9407-ed3a-4c5e-9368-6f2e6f5702bb service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:31.270393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 684/1371] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:31.539405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3e83f-e64f-49a7-b7df-49e9d87688ef None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.539832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3e83f-e64f-49a7-b7df-49e9d87688ef None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.539960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3e83f-e64f-49a7-b7df-49e9d87688ef None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.541605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 688/1372] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:31.547677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101908ef-0669-4205-81b6-f0b7f99e38c9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.547962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101908ef-0669-4205-81b6-f0b7f99e38c9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.548486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101908ef-0669-4205-81b6-f0b7f99e38c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.567529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-101908ef-0669-4205-81b6-f0b7f99e38c9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:31.568754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-101908ef-0669-4205-81b6-f0b7f99e38c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:31.577755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 685/1373] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:31.894188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6080498-9f54-422f-8c3e-f275386a148d None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.894477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6080498-9f54-422f-8c3e-f275386a148d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.894752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6080498-9f54-422f-8c3e-f275386a148d None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.896095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 689/1374] 10.222.0.73 () {54 vars in 963 bytes} [Thu May 12 17:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:31.901486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa68c93-c57a-4457-8033-84e63a4fd483 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:31.901750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa68c93-c57a-4457-8033-84e63a4fd483 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:31.902026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa68c93-c57a-4457-8033-84e63a4fd483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:31.949905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eaa68c93-c57a-4457-8033-84e63a4fd483 None None] MFA Rules not processed for user `71263b9cdb9d43ca9ce74c3e2069265f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:31.951084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eaa68c93-c57a-4457-8033-84e63a4fd483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:32.009000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 686/1375] 10.222.0.73 () {58 vars in 1060 bytes} [Thu May 12 17:37:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:32.016600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6080498-9f54-422f-8c3e-f275386a148d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:32.026899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6080498-9f54-422f-8c3e-f275386a148d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:32.030340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2dc21-bc91-40ee-9146-7530b00a0fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:32.030968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2dc21-bc91-40ee-9146-7530b00a0fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:32.031246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2dc21-bc91-40ee-9146-7530b00a0fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:32.031514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2dc21-bc91-40ee-9146-7530b00a0fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:32.040381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2dc21-bc91-40ee-9146-7530b00a0fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:32.041856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2dc21-bc91-40ee-9146-7530b00a0fc1 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:32.052145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 690/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:33.543003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4eb87-2e68-4651-8991-b7df460d1020 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:33.543619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4eb87-2e68-4651-8991-b7df460d1020 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:33.543619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4eb87-2e68-4651-8991-b7df460d1020 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:33.545387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 687/1377] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:33.551548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b5cc1-9286-4e87-928f-331cfc3d876e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:33.552138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b5cc1-9286-4e87-928f-331cfc3d876e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:33.552138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b5cc1-9286-4e87-928f-331cfc3d876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:33.571651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-404b5cc1-9286-4e87-928f-331cfc3d876e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:33.572921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-404b5cc1-9286-4e87-928f-331cfc3d876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:33.583842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 691/1378] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:33.627228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4eb87-2e68-4651-8991-b7df460d1020 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:33.642487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4eb87-2e68-4651-8991-b7df460d1020 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:33.647934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a164c8-1951-4307-b72b-b8351ba84ecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:33.648634 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a164c8-1951-4307-b72b-b8351ba84ecd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:33.648914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a164c8-1951-4307-b72b-b8351ba84ecd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:33.649183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a164c8-1951-4307-b72b-b8351ba84ecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:33.658519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a164c8-1951-4307-b72b-b8351ba84ecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:33.660009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a164c8-1951-4307-b72b-b8351ba84ecd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:33.680066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 688/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:35.975431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89bef6-69d8-46ce-a3c8-391ab6417380 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:35.975973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89bef6-69d8-46ce-a3c8-391ab6417380 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:35.975973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89bef6-69d8-46ce-a3c8-391ab6417380 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:35.978645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 692/1380] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:35.982860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680822f-88fc-4b41-a25b-9cbfe74b0593 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:35.983312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680822f-88fc-4b41-a25b-9cbfe74b0593 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:35.983421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680822f-88fc-4b41-a25b-9cbfe74b0593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:36.001324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2680822f-88fc-4b41-a25b-9cbfe74b0593 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:36.002499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2680822f-88fc-4b41-a25b-9cbfe74b0593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:36.010530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 689/1381] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:36.054207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89bef6-69d8-46ce-a3c8-391ab6417380 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:36.069503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89bef6-69d8-46ce-a3c8-391ab6417380 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:36.076094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5df10-f409-4bda-95b1-51f8bffb9717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:36.076885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5df10-f409-4bda-95b1-51f8bffb9717 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:36.077261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5df10-f409-4bda-95b1-51f8bffb9717 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:36.077652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5df10-f409-4bda-95b1-51f8bffb9717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:36.086292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5df10-f409-4bda-95b1-51f8bffb9717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:36.087945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5df10-f409-4bda-95b1-51f8bffb9717 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:36.097537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 693/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:38.407031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1f673-e219-4573-a3c9-13274d6c3996 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:38.407618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1f673-e219-4573-a3c9-13274d6c3996 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:38.407618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1f673-e219-4573-a3c9-13274d6c3996 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:38.409188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 690/1383] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:38.414682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0731bc-8d0e-4fb3-aa32-c717a2eaa8a6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:38.415525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0731bc-8d0e-4fb3-aa32-c717a2eaa8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:38.415810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0731bc-8d0e-4fb3-aa32-c717a2eaa8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:38.434379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2e0731bc-8d0e-4fb3-aa32-c717a2eaa8a6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:38.435571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2e0731bc-8d0e-4fb3-aa32-c717a2eaa8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:38.443433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 694/1384] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:38.485582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1f673-e219-4573-a3c9-13274d6c3996 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:38.500547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1f673-e219-4573-a3c9-13274d6c3996 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:38.506031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb213d67-36c2-45e9-a675-42bfea08e1d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:38.506858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb213d67-36c2-45e9-a675-42bfea08e1d1 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:38.507254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb213d67-36c2-45e9-a675-42bfea08e1d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:38.507620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb213d67-36c2-45e9-a675-42bfea08e1d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:38.516709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb213d67-36c2-45e9-a675-42bfea08e1d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:38.518353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb213d67-36c2-45e9-a675-42bfea08e1d1 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:38.528593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 691/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:37:40.796006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72c28c-5bcb-4196-a014-c998a127017c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:40.796859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72c28c-5bcb-4196-a014-c998a127017c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:40.796859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72c28c-5bcb-4196-a014-c998a127017c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:40.798057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 695/1386] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:37:40.804276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115077-f734-4d47-9d37-6d7e0aee5053 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:40.804736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115077-f734-4d47-9d37-6d7e0aee5053 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:40.805147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115077-f734-4d47-9d37-6d7e0aee5053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:40.824749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6c115077-f734-4d47-9d37-6d7e0aee5053 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:40.825926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c115077-f734-4d47-9d37-6d7e0aee5053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:40.836125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 692/1387] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:40.894547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72c28c-5bcb-4196-a014-c998a127017c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:40.910093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72c28c-5bcb-4196-a014-c998a127017c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:40.915655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b739-d898-4123-b59e-faffc72d6b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:40.916323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b739-d898-4123-b59e-faffc72d6b13 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:40.916640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b739-d898-4123-b59e-faffc72d6b13 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:40.917614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b739-d898-4123-b59e-faffc72d6b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:40.925862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b739-d898-4123-b59e-faffc72d6b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:40.927494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b739-d898-4123-b59e-faffc72d6b13 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:40.937139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 696/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:37:43.327696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c8de0-1533-441e-b883-34725b56caf1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:43.328258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c8de0-1533-441e-b883-34725b56caf1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:43.328381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c8de0-1533-441e-b883-34725b56caf1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:43.329855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 693/1389] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:43.335374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5330bb-c452-47a9-9519-a908c30a0dee None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:43.335771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5330bb-c452-47a9-9519-a908c30a0dee None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:43.335926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5330bb-c452-47a9-9519-a908c30a0dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:43.353705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7b5330bb-c452-47a9-9519-a908c30a0dee None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:43.355763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7b5330bb-c452-47a9-9519-a908c30a0dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:43.363735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 697/1390] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:37:43.409500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c8de0-1533-441e-b883-34725b56caf1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:43.425638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c8de0-1533-441e-b883-34725b56caf1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:43.431554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5db81-fa2c-43e6-971a-301359609a15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:43.432344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5db81-fa2c-43e6-971a-301359609a15 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:43.432935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5db81-fa2c-43e6-971a-301359609a15 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:43.433185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5db81-fa2c-43e6-971a-301359609a15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:43.442322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5db81-fa2c-43e6-971a-301359609a15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:43.444067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5db81-fa2c-43e6-971a-301359609a15 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:43.454362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 694/1391] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:46.002829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e969-4eda-47d1-89ac-58339a831046 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:46.003467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e969-4eda-47d1-89ac-58339a831046 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:46.003467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e969-4eda-47d1-89ac-58339a831046 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:46.004868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 698/1392] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:46.011132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2246-75ae-4e3f-9440-6f114e33ecff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:46.011690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2246-75ae-4e3f-9440-6f114e33ecff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:46.011690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2246-75ae-4e3f-9440-6f114e33ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:46.033289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-14ab2246-75ae-4e3f-9440-6f114e33ecff None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:46.034764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-14ab2246-75ae-4e3f-9440-6f114e33ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:46.044801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 695/1393] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:46.090514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3e969-4eda-47d1-89ac-58339a831046 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:46.126353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3e969-4eda-47d1-89ac-58339a831046 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:46.133256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a9c34-adae-4351-a7a4-0939992f504d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:46.133746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a9c34-adae-4351-a7a4-0939992f504d service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:46.134025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a9c34-adae-4351-a7a4-0939992f504d service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:46.134298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a9c34-adae-4351-a7a4-0939992f504d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:46.146142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a9c34-adae-4351-a7a4-0939992f504d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:46.148355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a9c34-adae-4351-a7a4-0939992f504d service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:46.159848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 699/1394] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:48.303542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19cc7d-0609-463f-8d8d-d46b2ac83037 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.304120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19cc7d-0609-463f-8d8d-d46b2ac83037 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.304120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19cc7d-0609-463f-8d8d-d46b2ac83037 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.305705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 696/1395] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:48.311880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a4441-070d-4097-b60e-6c4a3f33ea0c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.312495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a4441-070d-4097-b60e-6c4a3f33ea0c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.312600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a4441-070d-4097-b60e-6c4a3f33ea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.331830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6a8a4441-070d-4097-b60e-6c4a3f33ea0c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:48.333115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6a8a4441-070d-4097-b60e-6c4a3f33ea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:48.341185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 700/1396] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:48.397791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a7f18-aa17-4396-a5ca-7e06aa12d2e2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.398083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a7f18-aa17-4396-a5ca-7e06aa12d2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.398363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a7f18-aa17-4396-a5ca-7e06aa12d2e2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.399721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 697/1397] 10.222.0.73 () {54 vars in 962 bytes} [Thu May 12 17:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:48.405312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3dfa-9c63-484f-b10f-90565452a4e0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.405596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3dfa-9c63-484f-b10f-90565452a4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.405873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e3dfa-9c63-484f-b10f-90565452a4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.451321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0d3e3dfa-9c63-484f-b10f-90565452a4e0 None None] MFA Rules not processed for user `5621c3ccfb8040748f8adef3dc810c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:48.452524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0d3e3dfa-9c63-484f-b10f-90565452a4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:48.509013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa425bdd-8a7d-47d0-87a4-4c1b02239df3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.509298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa425bdd-8a7d-47d0-87a4-4c1b02239df3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.509573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa425bdd-8a7d-47d0-87a4-4c1b02239df3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.509875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 701/1398] 10.222.0.73 () {58 vars in 1059 bytes} [Thu May 12 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:48.511606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 698/1399] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:37:48.517081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.517378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.517660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.517777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa425bdd-8a7d-47d0-87a4-4c1b02239df3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:48.528848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa425bdd-8a7d-47d0-87a4-4c1b02239df3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:48.532325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00486f-ac72-445c-bd13-421af51d4d55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:48.532945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00486f-ac72-445c-bd13-421af51d4d55 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.533224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00486f-ac72-445c-bd13-421af51d4d55 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.533495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00486f-ac72-445c-bd13-421af51d4d55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.535350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:48.536584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:48.542469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00486f-ac72-445c-bd13-421af51d4d55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:48.543881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00486f-ac72-445c-bd13-421af51d4d55 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:48.544767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 702/1400] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:37:48.553050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 699/1401] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:37:48.589654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:48.601909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411fe59-a0b3-442f-9d80-17c4a5885ba1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:48.607782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70da83-99f7-457d-8417-1e67facf5cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:48.608447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70da83-99f7-457d-8417-1e67facf5cb2 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.608733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70da83-99f7-457d-8417-1e67facf5cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.609007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70da83-99f7-457d-8417-1e67facf5cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:48.621458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70da83-99f7-457d-8417-1e67facf5cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:48.623016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70da83-99f7-457d-8417-1e67facf5cb2 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:48.632663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 703/1402] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:37:48.987861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403f9-6293-4d98-804d-b8be531071b6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:48.988195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403f9-6293-4d98-804d-b8be531071b6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:48.988472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403f9-6293-4d98-804d-b8be531071b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:49.038110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-000403f9-6293-4d98-804d-b8be531071b6 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:49.039331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-000403f9-6293-4d98-804d-b8be531071b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:49.101255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 700/1403] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:49.148848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9c88b-a635-481d-a9e5-ae6639d8ddef None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:49.149210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9c88b-a635-481d-a9e5-ae6639d8ddef None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:49.149553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9c88b-a635-481d-a9e5-ae6639d8ddef None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:49.151707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 704/1404] 10.222.0.73 () {54 vars in 942 bytes} [Thu May 12 17:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:49.158159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6965-1d4e-41fe-93a9-897e99f7cba3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:49.158450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6965-1d4e-41fe-93a9-897e99f7cba3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:49.158723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6965-1d4e-41fe-93a9-897e99f7cba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:49.205360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ebad6965-1d4e-41fe-93a9-897e99f7cba3 None None] MFA Rules not processed for user `972f23cd94bf4051825701d2aaddfe6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:49.206563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ebad6965-1d4e-41fe-93a9-897e99f7cba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:49.275950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 701/1405] 10.222.0.73 () {58 vars in 1039 bytes} [Thu May 12 17:37:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:49.286115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9c88b-a635-481d-a9e5-ae6639d8ddef None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:49.299117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9c88b-a635-481d-a9e5-ae6639d8ddef None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:49.303948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190cacd-2cd8-4e94-9cd9-5410d7fc5089 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:49.304612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190cacd-2cd8-4e94-9cd9-5410d7fc5089 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:49.304896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190cacd-2cd8-4e94-9cd9-5410d7fc5089 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:49.305174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190cacd-2cd8-4e94-9cd9-5410d7fc5089 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:49.313825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190cacd-2cd8-4e94-9cd9-5410d7fc5089 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:49.315383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190cacd-2cd8-4e94-9cd9-5410d7fc5089 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:49.323839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 705/1406] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:37:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:37:50.901816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45818d-e6fb-4e85-ad94-5aff2eac0d2b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:50.902385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45818d-e6fb-4e85-ad94-5aff2eac0d2b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:50.902385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45818d-e6fb-4e85-ad94-5aff2eac0d2b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:50.903673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 702/1407] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:37:50.909208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006ba0-98a8-4d4f-a79a-8724e154d93d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:50.909893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006ba0-98a8-4d4f-a79a-8724e154d93d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:50.909893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006ba0-98a8-4d4f-a79a-8724e154d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:50.928286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b9006ba0-98a8-4d4f-a79a-8724e154d93d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:37:50.929707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b9006ba0-98a8-4d4f-a79a-8724e154d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:37:50.939443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 706/1408] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:37:50.983993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45818d-e6fb-4e85-ad94-5aff2eac0d2b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:37:50.998838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45818d-e6fb-4e85-ad94-5aff2eac0d2b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:37:51.004078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff35f9-bced-47fc-bab6-dc478d1387da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:37:51.004742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff35f9-bced-47fc-bab6-dc478d1387da service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:37:51.005019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff35f9-bced-47fc-bab6-dc478d1387da service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:37:51.005290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff35f9-bced-47fc-bab6-dc478d1387da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:37:51.015565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff35f9-bced-47fc-bab6-dc478d1387da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:37:51.017154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff35f9-bced-47fc-bab6-dc478d1387da service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:37:51.027783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 703/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:12.736216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654a9c1-16c9-4c91-8dba-c200e975d1b9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:12.736864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654a9c1-16c9-4c91-8dba-c200e975d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:12.736864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654a9c1-16c9-4c91-8dba-c200e975d1b9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:12.738118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 707/1410] 10.222.0.73 () {54 vars in 918 bytes} [Thu May 12 17:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:12.743665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204fae-75d3-49d0-b1c4-d96b6d4a4558 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:12.744148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204fae-75d3-49d0-b1c4-d96b6d4a4558 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:12.744276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204fae-75d3-49d0-b1c4-d96b6d4a4558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:12.762345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-87204fae-75d3-49d0-b1c4-d96b6d4a4558 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:12.763529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-87204fae-75d3-49d0-b1c4-d96b6d4a4558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:12.771877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 704/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Thu May 12 17:38:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:12.786372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd166fd-0437-465d-8d13-76c57d02207c None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:12.786662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd166fd-0437-465d-8d13-76c57d02207c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:12.786941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd166fd-0437-465d-8d13-76c57d02207c None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:12.788299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 708/1412] 10.222.0.73 () {54 vars in 918 bytes} [Thu May 12 17:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:38:12.793536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee740-c828-4ba7-929f-9f444b209c78 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:12.793817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee740-c828-4ba7-929f-9f444b209c78 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:12.794088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee740-c828-4ba7-929f-9f444b209c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:12.811209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8fbee740-c828-4ba7-929f-9f444b209c78 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:12.812401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8fbee740-c828-4ba7-929f-9f444b209c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:12.821226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 705/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Thu May 12 17:38:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:38:15.507921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870983-70fa-4c7a-b645-ceed10d26877 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:15.508623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870983-70fa-4c7a-b645-ceed10d26877 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:15.508623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870983-70fa-4c7a-b645-ceed10d26877 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:15.509899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:15.515208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed98ae-de22-44e7-9db9-26c167e45aec None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:15.515630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed98ae-de22-44e7-9db9-26c167e45aec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:15.515747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed98ae-de22-44e7-9db9-26c167e45aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:15.533278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9bed98ae-de22-44e7-9db9-26c167e45aec None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:15.534451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9bed98ae-de22-44e7-9db9-26c167e45aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:15.542993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:15.583015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870983-70fa-4c7a-b645-ceed10d26877 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:15.597619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870983-70fa-4c7a-b645-ceed10d26877 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:15.602824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d39634-d8f8-4946-91f5-43b06594a7cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:15.603453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39634-d8f8-4946-91f5-43b06594a7cb service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:15.603733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39634-d8f8-4946-91f5-43b06594a7cb service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:15.604001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39634-d8f8-4946-91f5-43b06594a7cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:15.612587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d39634-d8f8-4946-91f5-43b06594a7cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:15.614151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d39634-d8f8-4946-91f5-43b06594a7cb service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:15.623751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 710/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:16.614057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eae34-dde3-4167-8157-95f0da6fb53c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:16.614629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eae34-dde3-4167-8157-95f0da6fb53c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:16.614629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eae34-dde3-4167-8157-95f0da6fb53c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:16.615967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:16.622431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeef783-e216-485d-ad00-5cf086d86603 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:16.622889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeef783-e216-485d-ad00-5cf086d86603 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:16.622993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeef783-e216-485d-ad00-5cf086d86603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:16.641399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-edeef783-e216-485d-ad00-5cf086d86603 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:16.642588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-edeef783-e216-485d-ad00-5cf086d86603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:16.650262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:16.821335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eae34-dde3-4167-8157-95f0da6fb53c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:16.835750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eae34-dde3-4167-8157-95f0da6fb53c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:16.841625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece7830-1b72-4c96-ab06-5edcb9352fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:16.842257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece7830-1b72-4c96-ab06-5edcb9352fab service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:16.842539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece7830-1b72-4c96-ab06-5edcb9352fab service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:16.842814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece7830-1b72-4c96-ab06-5edcb9352fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:16.851280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece7830-1b72-4c96-ab06-5edcb9352fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:16.852895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece7830-1b72-4c96-ab06-5edcb9352fab service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:16.863172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 708/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:17.806675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2c43b-150d-4901-8717-b36ce11ed934 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:17.807753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2c43b-150d-4901-8717-b36ce11ed934 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:17.808332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2c43b-150d-4901-8717-b36ce11ed934 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:17.809715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:17.814706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d86a4b-9a7e-426d-9522-5f30b5e8fe75 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:17.815131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d86a4b-9a7e-426d-9522-5f30b5e8fe75 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:17.815238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d86a4b-9a7e-426d-9522-5f30b5e8fe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:17.833923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-07d86a4b-9a7e-426d-9522-5f30b5e8fe75 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:17.835110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07d86a4b-9a7e-426d-9522-5f30b5e8fe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:17.842905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:17.998008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2c43b-150d-4901-8717-b36ce11ed934 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:18.012369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2c43b-150d-4901-8717-b36ce11ed934 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:18.017665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1f2ef-ac42-4c3a-b4c0-75e4c9cfba28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:18.018302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1f2ef-ac42-4c3a-b4c0-75e4c9cfba28 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:18.018584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1f2ef-ac42-4c3a-b4c0-75e4c9cfba28 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:18.018863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1f2ef-ac42-4c3a-b4c0-75e4c9cfba28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:18.027723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1f2ef-ac42-4c3a-b4c0-75e4c9cfba28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:18.029316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1f2ef-ac42-4c3a-b4c0-75e4c9cfba28 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:18.038373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 713/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:18.982742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4c16-f7ea-4f01-929b-ba4450d31e0d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:18.983315 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4c16-f7ea-4f01-929b-ba4450d31e0d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:18.983315 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4c16-f7ea-4f01-929b-ba4450d31e0d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:18.984696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:18.990490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943c14-3c95-4669-ae3c-fe667b106e53 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:18.990906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943c14-3c95-4669-ae3c-fe667b106e53 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:18.991048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943c14-3c95-4669-ae3c-fe667b106e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:19.011379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-67943c14-3c95-4669-ae3c-fe667b106e53 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:19.012643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-67943c14-3c95-4669-ae3c-fe667b106e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:19.020992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:19.175516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c4c16-f7ea-4f01-929b-ba4450d31e0d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:19.189812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c4c16-f7ea-4f01-929b-ba4450d31e0d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:19.195361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46626-8785-4706-9247-2fa49c09d30e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:19.195997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46626-8785-4706-9247-2fa49c09d30e service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:19.196323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46626-8785-4706-9247-2fa49c09d30e service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:19.196599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46626-8785-4706-9247-2fa49c09d30e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:19.205111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46626-8785-4706-9247-2fa49c09d30e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:19.206730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46626-8785-4706-9247-2fa49c09d30e service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:19.216390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 711/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:20.140586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ab47-6c91-4c7a-9e7b-81e951a2ea35 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:20.141155 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ab47-6c91-4c7a-9e7b-81e951a2ea35 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:20.141155 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ab47-6c91-4c7a-9e7b-81e951a2ea35 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:20.142485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:20.147326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d1e9-392f-49e8-846f-1c0de59d21b2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:20.147718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d1e9-392f-49e8-846f-1c0de59d21b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:20.147884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d1e9-392f-49e8-846f-1c0de59d21b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:20.166140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-69c4d1e9-392f-49e8-846f-1c0de59d21b2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:20.168236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-69c4d1e9-392f-49e8-846f-1c0de59d21b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:20.175882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:20.329249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ab47-6c91-4c7a-9e7b-81e951a2ea35 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:20.343460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ab47-6c91-4c7a-9e7b-81e951a2ea35 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:20.348718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb7da8-6748-49ec-afbe-be0efd2684b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:20.349346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7da8-6748-49ec-afbe-be0efd2684b6 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:20.349623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7da8-6748-49ec-afbe-be0efd2684b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:20.349893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7da8-6748-49ec-afbe-be0efd2684b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:20.358360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb7da8-6748-49ec-afbe-be0efd2684b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:20.359903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb7da8-6748-49ec-afbe-be0efd2684b6 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:20.368778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 716/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:21.246338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac31e5-6e29-4372-8fd6-fddb2cae9ed4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:21.246908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac31e5-6e29-4372-8fd6-fddb2cae9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:21.246908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac31e5-6e29-4372-8fd6-fddb2cae9ed4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:21.248187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:21.253873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729589dc-e2c5-4465-b9e0-369b650ecda2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:21.254354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729589dc-e2c5-4465-b9e0-369b650ecda2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:21.254451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729589dc-e2c5-4465-b9e0-369b650ecda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:21.272470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-729589dc-e2c5-4465-b9e0-369b650ecda2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:21.273663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-729589dc-e2c5-4465-b9e0-369b650ecda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:21.281361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:21.430643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac31e5-6e29-4372-8fd6-fddb2cae9ed4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:21.445107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac31e5-6e29-4372-8fd6-fddb2cae9ed4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:21.450331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b298b-d210-4178-b822-1fcc1230a59b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:21.450947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b298b-d210-4178-b822-1fcc1230a59b service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:21.451222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b298b-d210-4178-b822-1fcc1230a59b service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:21.451489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b298b-d210-4178-b822-1fcc1230a59b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:21.459162 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b298b-d210-4178-b822-1fcc1230a59b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:21.460750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b298b-d210-4178-b822-1fcc1230a59b service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:21.469697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 714/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:38:22.421475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef92b-99e9-4545-b585-d4cf245f5009 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:22.422026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef92b-99e9-4545-b585-d4cf245f5009 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:22.422026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef92b-99e9-4545-b585-d4cf245f5009 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:22.423567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:22.429233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09caf8-a221-4be3-8620-82ffab7507af None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:22.429694 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09caf8-a221-4be3-8620-82ffab7507af None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:22.429804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09caf8-a221-4be3-8620-82ffab7507af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:22.449980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ff09caf8-a221-4be3-8620-82ffab7507af None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:22.451211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ff09caf8-a221-4be3-8620-82ffab7507af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:22.459372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:38:22.611046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cef92b-99e9-4545-b585-d4cf245f5009 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:22.626551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cef92b-99e9-4545-b585-d4cf245f5009 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:22.631904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7690f-9a6d-4e26-8276-7b7b6897fda4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:22.632578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7690f-9a6d-4e26-8276-7b7b6897fda4 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:22.632864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7690f-9a6d-4e26-8276-7b7b6897fda4 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:22.633143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7690f-9a6d-4e26-8276-7b7b6897fda4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:22.641903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7690f-9a6d-4e26-8276-7b7b6897fda4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:22.643462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7690f-9a6d-4e26-8276-7b7b6897fda4 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:22.652629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 719/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:23.528728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc97c3-3f52-4a86-ab9a-3d723a9f49eb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:23.529299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc97c3-3f52-4a86-ab9a-3d723a9f49eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:23.529574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc97c3-3f52-4a86-ab9a-3d723a9f49eb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:23.530960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:23.535939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01248f-eab7-4e83-8192-f8aa6cfb4288 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:23.536342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01248f-eab7-4e83-8192-f8aa6cfb4288 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:23.536538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01248f-eab7-4e83-8192-f8aa6cfb4288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:23.553733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6e01248f-eab7-4e83-8192-f8aa6cfb4288 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:23.554902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6e01248f-eab7-4e83-8192-f8aa6cfb4288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:23.562595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:23.717767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc97c3-3f52-4a86-ab9a-3d723a9f49eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:23.732484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc97c3-3f52-4a86-ab9a-3d723a9f49eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:23.737783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f12af7-239d-4cec-9dfb-ca40d8a44ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:23.738406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f12af7-239d-4cec-9dfb-ca40d8a44ef5 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:23.738684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f12af7-239d-4cec-9dfb-ca40d8a44ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:23.738952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f12af7-239d-4cec-9dfb-ca40d8a44ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:23.747691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f12af7-239d-4cec-9dfb-ca40d8a44ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:23.749360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f12af7-239d-4cec-9dfb-ca40d8a44ef5 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:23.758341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 717/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:24.634982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472e62-f3c3-49a2-8c28-a8926c3b50bb None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:24.635714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472e62-f3c3-49a2-8c28-a8926c3b50bb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:24.636021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472e62-f3c3-49a2-8c28-a8926c3b50bb None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:24.637461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:24.642338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bf6eb-3b98-4928-bd01-c6b342a27b01 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:24.642786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bf6eb-3b98-4928-bd01-c6b342a27b01 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:24.642886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bf6eb-3b98-4928-bd01-c6b342a27b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:24.663513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-863bf6eb-3b98-4928-bd01-c6b342a27b01 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:24.664741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-863bf6eb-3b98-4928-bd01-c6b342a27b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:24.673320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:24.820656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472e62-f3c3-49a2-8c28-a8926c3b50bb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:24.835147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472e62-f3c3-49a2-8c28-a8926c3b50bb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:24.841594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e635d-ec09-42dc-b0ab-504b912ca5de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:24.842217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e635d-ec09-42dc-b0ab-504b912ca5de service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:24.842501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e635d-ec09-42dc-b0ab-504b912ca5de service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:24.842775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e635d-ec09-42dc-b0ab-504b912ca5de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:24.850838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e635d-ec09-42dc-b0ab-504b912ca5de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:24.852430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e635d-ec09-42dc-b0ab-504b912ca5de service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:24.863036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 722/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:26.953657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc0609-0932-42ef-9668-499fea63499c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:26.954145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc0609-0932-42ef-9668-499fea63499c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:26.954240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc0609-0932-42ef-9668-499fea63499c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:26.955570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:26.961074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8c3d6-38b4-44f1-9cf8-e459a1b43b2b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:26.961510 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8c3d6-38b4-44f1-9cf8-e459a1b43b2b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:26.961619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8c3d6-38b4-44f1-9cf8-e459a1b43b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:26.980894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d4b8c3d6-38b4-44f1-9cf8-e459a1b43b2b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:26.982093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d4b8c3d6-38b4-44f1-9cf8-e459a1b43b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:26.989823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:27.028267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc0609-0932-42ef-9668-499fea63499c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:27.038581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc0609-0932-42ef-9668-499fea63499c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:27.042906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ed890-59cd-468a-b72b-785b9b43519e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:27.043535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed890-59cd-468a-b72b-785b9b43519e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:27.043811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed890-59cd-468a-b72b-785b9b43519e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:27.044081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed890-59cd-468a-b72b-785b9b43519e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:27.052646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed890-59cd-468a-b72b-785b9b43519e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:27.054063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed890-59cd-468a-b72b-785b9b43519e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:27.063182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 720/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:29.434043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d9ba2-7278-4882-8d72-6599c5884fa1 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:29.434575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d9ba2-7278-4882-8d72-6599c5884fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:29.434575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d9ba2-7278-4882-8d72-6599c5884fa1 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:29.435939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:29.441135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642185d-91fb-4e9a-898b-5869a69db406 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:29.441547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642185d-91fb-4e9a-898b-5869a69db406 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:29.441687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642185d-91fb-4e9a-898b-5869a69db406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:29.458875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5642185d-91fb-4e9a-898b-5869a69db406 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:29.460078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5642185d-91fb-4e9a-898b-5869a69db406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:29.467691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:29.506264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d9ba2-7278-4882-8d72-6599c5884fa1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:29.507932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-642d9ba2-7278-4882-8d72-6599c5884fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:29.520628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d9ba2-7278-4882-8d72-6599c5884fa1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:29.524047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e135c-f63f-45e0-8cc7-091e58b08d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:29.524705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e135c-f63f-45e0-8cc7-091e58b08d64 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:29.524989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e135c-f63f-45e0-8cc7-091e58b08d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:29.525265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e135c-f63f-45e0-8cc7-091e58b08d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:29.533983 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e135c-f63f-45e0-8cc7-091e58b08d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:29.535386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e135c-f63f-45e0-8cc7-091e58b08d64 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:29.546255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 725/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:32.096024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3f6e7-6248-48d1-9644-016d0cde95c2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:32.096580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3f6e7-6248-48d1-9644-016d0cde95c2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:32.096580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3f6e7-6248-48d1-9644-016d0cde95c2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:32.097957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:32.103376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709f0e6-2b4d-49e1-a349-490d02fbf787 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:32.104371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709f0e6-2b4d-49e1-a349-490d02fbf787 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:32.104547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709f0e6-2b4d-49e1-a349-490d02fbf787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:32.122022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3709f0e6-2b4d-49e1-a349-490d02fbf787 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:32.123191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3709f0e6-2b4d-49e1-a349-490d02fbf787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:32.130801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:32.169963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3f6e7-6248-48d1-9644-016d0cde95c2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:32.180268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3f6e7-6248-48d1-9644-016d0cde95c2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:32.183827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f8b54-a9cc-41a3-b6b2-fcc6d9a60221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:32.184483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f8b54-a9cc-41a3-b6b2-fcc6d9a60221 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:32.184764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f8b54-a9cc-41a3-b6b2-fcc6d9a60221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:32.185039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f8b54-a9cc-41a3-b6b2-fcc6d9a60221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:32.193546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f8b54-a9cc-41a3-b6b2-fcc6d9a60221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:32.194959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f8b54-a9cc-41a3-b6b2-fcc6d9a60221 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:32.204148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 723/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:34.540332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a999b4d-d462-4e22-92ae-e9a3fbebb5bc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:34.540964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a999b4d-d462-4e22-92ae-e9a3fbebb5bc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:34.540964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a999b4d-d462-4e22-92ae-e9a3fbebb5bc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:34.542879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:34.548101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1b2dc-9af8-46c4-9335-d887e878d83a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:34.548536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1b2dc-9af8-46c4-9335-d887e878d83a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:34.548690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1b2dc-9af8-46c4-9335-d887e878d83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:34.566467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6af1b2dc-9af8-46c4-9335-d887e878d83a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:34.567643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6af1b2dc-9af8-46c4-9335-d887e878d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:34.575941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:34.612716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a999b4d-d462-4e22-92ae-e9a3fbebb5bc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:34.624077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a999b4d-d462-4e22-92ae-e9a3fbebb5bc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:34.627494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1b24d-0767-45f6-b51d-60b29cd551b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:34.628122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1b24d-0767-45f6-b51d-60b29cd551b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:34.628436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1b24d-0767-45f6-b51d-60b29cd551b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:34.628709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1b24d-0767-45f6-b51d-60b29cd551b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:34.638074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1b24d-0767-45f6-b51d-60b29cd551b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:34.639482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1b24d-0767-45f6-b51d-60b29cd551b5 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:34.648542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 728/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:36.903658 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9be0a7-5782-4180-b2c1-c5e308355dda None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:36.904211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9be0a7-5782-4180-b2c1-c5e308355dda None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:36.904211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9be0a7-5782-4180-b2c1-c5e308355dda None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:36.905584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:36.910776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1bc80-3163-4465-a152-41732d1d40d8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:36.911214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1bc80-3163-4465-a152-41732d1d40d8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:36.911355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1bc80-3163-4465-a152-41732d1d40d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:36.929557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-97e1bc80-3163-4465-a152-41732d1d40d8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:36.930730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-97e1bc80-3163-4465-a152-41732d1d40d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:36.939482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:36.975989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9be0a7-5782-4180-b2c1-c5e308355dda None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:36.986868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9be0a7-5782-4180-b2c1-c5e308355dda None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:36.990325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aed75-e909-4a45-a73c-ca99c067461c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:36.990954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aed75-e909-4a45-a73c-ca99c067461c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:36.991233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aed75-e909-4a45-a73c-ca99c067461c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:36.991506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aed75-e909-4a45-a73c-ca99c067461c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:37.000899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2aed75-e909-4a45-a73c-ca99c067461c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:37.002308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2aed75-e909-4a45-a73c-ca99c067461c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:37.011257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 726/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:39.299657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf9edb-39cb-4e3b-8e0d-53ab49be36d0 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:39.300206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf9edb-39cb-4e3b-8e0d-53ab49be36d0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:39.300206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf9edb-39cb-4e3b-8e0d-53ab49be36d0 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:39.301495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:39.306901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1b6ef-4cc3-47a2-881f-cfb2e19a9fa9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:39.307310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1b6ef-4cc3-47a2-881f-cfb2e19a9fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:39.307475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1b6ef-4cc3-47a2-881f-cfb2e19a9fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:39.325195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0ef1b6ef-4cc3-47a2-881f-cfb2e19a9fa9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:39.326400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0ef1b6ef-4cc3-47a2-881f-cfb2e19a9fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:39.334014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:39.371933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf9edb-39cb-4e3b-8e0d-53ab49be36d0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:39.382161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf9edb-39cb-4e3b-8e0d-53ab49be36d0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:39.385564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a348f71-9adb-42c4-87bb-12f77d130383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:39.386184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a348f71-9adb-42c4-87bb-12f77d130383 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:39.386459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a348f71-9adb-42c4-87bb-12f77d130383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:39.386732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a348f71-9adb-42c4-87bb-12f77d130383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:39.395092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a348f71-9adb-42c4-87bb-12f77d130383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:39.396520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a348f71-9adb-42c4-87bb-12f77d130383 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:39.405544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 731/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:41.665989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0441a6-6220-4a47-a1a2-89e2454e9db1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:41.666530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0441a6-6220-4a47-a1a2-89e2454e9db1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:41.666530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0441a6-6220-4a47-a1a2-89e2454e9db1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:41.668482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:41.673987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038755a-2392-4076-9885-924575889da2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:41.674425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038755a-2392-4076-9885-924575889da2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:41.674542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038755a-2392-4076-9885-924575889da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:41.692037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3038755a-2392-4076-9885-924575889da2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:41.693235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3038755a-2392-4076-9885-924575889da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:41.700880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:41.738551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0441a6-6220-4a47-a1a2-89e2454e9db1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:41.749134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0441a6-6220-4a47-a1a2-89e2454e9db1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:41.752555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f7689-10b3-4d65-8c99-410d981d73ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:41.753178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f7689-10b3-4d65-8c99-410d981d73ba service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:41.753455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f7689-10b3-4d65-8c99-410d981d73ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:41.753729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f7689-10b3-4d65-8c99-410d981d73ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:41.762200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f7689-10b3-4d65-8c99-410d981d73ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:41.763610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f7689-10b3-4d65-8c99-410d981d73ba service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:41.772637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 729/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:44.067556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc29a7-b3f0-47f4-9f0b-b989d421e222 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:44.068101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc29a7-b3f0-47f4-9f0b-b989d421e222 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:44.068101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc29a7-b3f0-47f4-9f0b-b989d421e222 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:44.069501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:44.075301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a789f27-bf9d-4c17-9cd5-47789f347d3b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:44.075703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a789f27-bf9d-4c17-9cd5-47789f347d3b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:44.075854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a789f27-bf9d-4c17-9cd5-47789f347d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:44.093829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4a789f27-bf9d-4c17-9cd5-47789f347d3b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:44.095018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a789f27-bf9d-4c17-9cd5-47789f347d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:44.102686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:44.139664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc29a7-b3f0-47f4-9f0b-b989d421e222 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:44.150143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc29a7-b3f0-47f4-9f0b-b989d421e222 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:44.154567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c01459-61f8-4faf-9e13-36fa6eb8ccbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:44.155188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c01459-61f8-4faf-9e13-36fa6eb8ccbe service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:44.155462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c01459-61f8-4faf-9e13-36fa6eb8ccbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:44.155729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c01459-61f8-4faf-9e13-36fa6eb8ccbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:44.164267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c01459-61f8-4faf-9e13-36fa6eb8ccbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:44.165591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c01459-61f8-4faf-9e13-36fa6eb8ccbe service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:44.175005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 734/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:46.420533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1231-74b2-4e2c-aef5-1c931c5163f2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:46.421076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1231-74b2-4e2c-aef5-1c931c5163f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:46.421201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1231-74b2-4e2c-aef5-1c931c5163f2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:46.422468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:46.427758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28b421-5a56-42a8-b16f-568016b96fc5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:46.428202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28b421-5a56-42a8-b16f-568016b96fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:46.428327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28b421-5a56-42a8-b16f-568016b96fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:46.446043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9e28b421-5a56-42a8-b16f-568016b96fc5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:46.447219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9e28b421-5a56-42a8-b16f-568016b96fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:46.455368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:46.490118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd1231-74b2-4e2c-aef5-1c931c5163f2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:46.501400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd1231-74b2-4e2c-aef5-1c931c5163f2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:46.504851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e60c08-23dc-4da2-ba50-8a24f8360910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:46.505475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e60c08-23dc-4da2-ba50-8a24f8360910 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:46.505892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e60c08-23dc-4da2-ba50-8a24f8360910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:46.506176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e60c08-23dc-4da2-ba50-8a24f8360910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:46.515382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e60c08-23dc-4da2-ba50-8a24f8360910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:46.516867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e60c08-23dc-4da2-ba50-8a24f8360910 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:46.525833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 732/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:38:48.751308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d6370-c22e-400a-af46-39342e4a12ca None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:48.751865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d6370-c22e-400a-af46-39342e4a12ca None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:48.751865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d6370-c22e-400a-af46-39342e4a12ca None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:48.753262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:38:48.758702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9de6ab-923e-4348-9d49-4bdfdc745877 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:48.759119 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9de6ab-923e-4348-9d49-4bdfdc745877 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:48.759243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9de6ab-923e-4348-9d49-4bdfdc745877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:48.777547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6a9de6ab-923e-4348-9d49-4bdfdc745877 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:38:48.780474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6a9de6ab-923e-4348-9d49-4bdfdc745877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:38:48.788190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:38:48.832154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d6370-c22e-400a-af46-39342e4a12ca None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:38:48.843381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d6370-c22e-400a-af46-39342e4a12ca None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:38:48.847099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022c46b-cacf-4d6e-8fbe-60098a5636ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:38:48.847737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022c46b-cacf-4d6e-8fbe-60098a5636ce service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:38:48.848017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022c46b-cacf-4d6e-8fbe-60098a5636ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:38:48.848326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022c46b-cacf-4d6e-8fbe-60098a5636ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:38:48.857841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022c46b-cacf-4d6e-8fbe-60098a5636ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:38:48.859253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022c46b-cacf-4d6e-8fbe-60098a5636ce service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:38:48.868430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 737/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:39:22.003828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e278-ba76-4a75-aef5-dc74a5e0876e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:22.004522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e278-ba76-4a75-aef5-dc74a5e0876e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:22.004522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e278-ba76-4a75-aef5-dc74a5e0876e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:22.005803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 734/1471] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:22.011477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a22b3-9a3d-46ce-8f3d-a91f18b82431 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:22.011929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a22b3-9a3d-46ce-8f3d-a91f18b82431 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:22.012033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a22b3-9a3d-46ce-8f3d-a91f18b82431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:22.029602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-557a22b3-9a3d-46ce-8f3d-a91f18b82431 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:22.030765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-557a22b3-9a3d-46ce-8f3d-a91f18b82431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:22.038224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 738/1472] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:39:22.083426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbab2f-9237-4e6e-a6a3-90433202c95e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:22.083713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbab2f-9237-4e6e-a6a3-90433202c95e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:22.083991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbab2f-9237-4e6e-a6a3-90433202c95e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:22.085392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 735/1473] 10.222.0.73 () {54 vars in 962 bytes} [Thu May 12 17:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:22.091203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75d078-ada0-46df-9dda-88bdc51c2445 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:22.091494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75d078-ada0-46df-9dda-88bdc51c2445 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:22.091772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75d078-ada0-46df-9dda-88bdc51c2445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:22.111540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6f75d078-ada0-46df-9dda-88bdc51c2445 None None] MFA Rules not processed for user `5621c3ccfb8040748f8adef3dc810c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:22.112775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6f75d078-ada0-46df-9dda-88bdc51c2445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:22.119151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 739/1474] 10.222.0.73 () {58 vars in 1059 bytes} [Thu May 12 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:22.126808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbab2f-9237-4e6e-a6a3-90433202c95e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:22.138089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbab2f-9237-4e6e-a6a3-90433202c95e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:22.141531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1159ba-e97c-49b5-8821-2a58b280c9e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:22.142160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1159ba-e97c-49b5-8821-2a58b280c9e1 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:22.142442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1159ba-e97c-49b5-8821-2a58b280c9e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:22.142717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1159ba-e97c-49b5-8821-2a58b280c9e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:22.152340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1159ba-e97c-49b5-8821-2a58b280c9e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:22.153754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1159ba-e97c-49b5-8821-2a58b280c9e1 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:22.163014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 736/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:39:24.550055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cbb28-c707-4bf0-9f41-2c6fc2958ada None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:24.550620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cbb28-c707-4bf0-9f41-2c6fc2958ada None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:24.550620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cbb28-c707-4bf0-9f41-2c6fc2958ada None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:24.551944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 740/1476] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:24.557581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45af512-37ba-46a4-8c6f-35c75a63ab0b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:24.558114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45af512-37ba-46a4-8c6f-35c75a63ab0b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:24.558114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45af512-37ba-46a4-8c6f-35c75a63ab0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:24.575937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a45af512-37ba-46a4-8c6f-35c75a63ab0b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:24.577168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a45af512-37ba-46a4-8c6f-35c75a63ab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:24.584850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 737/1477] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:24.608322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cbb28-c707-4bf0-9f41-2c6fc2958ada None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:24.621673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cbb28-c707-4bf0-9f41-2c6fc2958ada None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:24.626069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:24.626736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:24.627022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:24.627293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:24.628018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:24.629779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:39:24.629779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:39:24.629779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:39:24.630911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bb94c-2a29-42c0-ad0d-e42210660fbb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:24.651508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 741/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Thu May 12 17:39:24 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:39:26.901619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae410f87-caf1-44d4-abb0-823a2bd86d80 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:26.902135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae410f87-caf1-44d4-abb0-823a2bd86d80 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:26.902135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae410f87-caf1-44d4-abb0-823a2bd86d80 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:26.903433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 738/1479] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:26.908598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbf91-0a9a-4f0c-a77f-192d728bd7a4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:26.908999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbf91-0a9a-4f0c-a77f-192d728bd7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:26.909144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbf91-0a9a-4f0c-a77f-192d728bd7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:26.926937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-aeddbf91-0a9a-4f0c-a77f-192d728bd7a4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:26.928121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aeddbf91-0a9a-4f0c-a77f-192d728bd7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:26.935855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 742/1480] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:26.976887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae410f87-caf1-44d4-abb0-823a2bd86d80 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:27.003523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae410f87-caf1-44d4-abb0-823a2bd86d80 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:27.009226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688db47-530f-4e6c-987d-b8ae9e94e3d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:27.009907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688db47-530f-4e6c-987d-b8ae9e94e3d6 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:27.010187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688db47-530f-4e6c-987d-b8ae9e94e3d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:27.010625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688db47-530f-4e6c-987d-b8ae9e94e3d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:27.019335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688db47-530f-4e6c-987d-b8ae9e94e3d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:27.020938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688db47-530f-4e6c-987d-b8ae9e94e3d6 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:27.031035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 739/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:39:29.432939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bbcf5-8cc0-4057-b20d-a476e206e836 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:29.433490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bbcf5-8cc0-4057-b20d-a476e206e836 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:29.433591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bbcf5-8cc0-4057-b20d-a476e206e836 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:29.434887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 743/1482] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:29.440049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5766a-a016-40a4-8461-fa9286999f17 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:29.440494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5766a-a016-40a4-8461-fa9286999f17 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:29.440653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5766a-a016-40a4-8461-fa9286999f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:29.457681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-18f5766a-a016-40a4-8461-fa9286999f17 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:29.458867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-18f5766a-a016-40a4-8461-fa9286999f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:29.466756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 740/1483] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:29.502847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bbcf5-8cc0-4057-b20d-a476e206e836 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:29.513529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bbcf5-8cc0-4057-b20d-a476e206e836 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:29.516979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1a7cd-5a9e-4035-bd40-be2a559db691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:29.517585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1a7cd-5a9e-4035-bd40-be2a559db691 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:29.517867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1a7cd-5a9e-4035-bd40-be2a559db691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:29.518143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1a7cd-5a9e-4035-bd40-be2a559db691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:29.526749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1a7cd-5a9e-4035-bd40-be2a559db691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:29.528145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1a7cd-5a9e-4035-bd40-be2a559db691 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:29.537340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 744/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:39:31.815778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc401c2-01c9-4212-883c-64062401b54e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:31.816375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc401c2-01c9-4212-883c-64062401b54e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:31.816375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc401c2-01c9-4212-883c-64062401b54e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:31.817721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 741/1485] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:31.823347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7bea-b380-4caf-98c6-0f09e4119fc0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:31.823736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7bea-b380-4caf-98c6-0f09e4119fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:31.823898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7bea-b380-4caf-98c6-0f09e4119fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:31.841414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b9fb7bea-b380-4caf-98c6-0f09e4119fc0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:31.842585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b9fb7bea-b380-4caf-98c6-0f09e4119fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:31.850177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 745/1486] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:31.890909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc401c2-01c9-4212-883c-64062401b54e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:31.902166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc401c2-01c9-4212-883c-64062401b54e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:31.905582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbb3a-9e5c-4404-8b53-6fb69a49e115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:31.906224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbb3a-9e5c-4404-8b53-6fb69a49e115 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:31.906501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbb3a-9e5c-4404-8b53-6fb69a49e115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:31.906769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbb3a-9e5c-4404-8b53-6fb69a49e115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:31.915451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacbb3a-9e5c-4404-8b53-6fb69a49e115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:31.917123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacbb3a-9e5c-4404-8b53-6fb69a49e115 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:31.926266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 742/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:39:34.277317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e0feb-0d38-4af1-b823-1dbf3c6ae3d2 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:34.278022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e0feb-0d38-4af1-b823-1dbf3c6ae3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:34.278022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e0feb-0d38-4af1-b823-1dbf3c6ae3d2 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:34.279203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 746/1488] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:34.284359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da9c2-ec3b-4293-95d8-505106144e7a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:34.284762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da9c2-ec3b-4293-95d8-505106144e7a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:34.284916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da9c2-ec3b-4293-95d8-505106144e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:34.301951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fa5da9c2-ec3b-4293-95d8-505106144e7a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:34.303133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fa5da9c2-ec3b-4293-95d8-505106144e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:34.311370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 743/1489] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:34.351081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e0feb-0d38-4af1-b823-1dbf3c6ae3d2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:34.361654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e0feb-0d38-4af1-b823-1dbf3c6ae3d2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:34.365119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d567be1-1644-428f-97ab-e6634a17e1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:34.365746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567be1-1644-428f-97ab-e6634a17e1aa service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:34.366053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567be1-1644-428f-97ab-e6634a17e1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:34.366326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567be1-1644-428f-97ab-e6634a17e1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:34.375685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d567be1-1644-428f-97ab-e6634a17e1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:34.377140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d567be1-1644-428f-97ab-e6634a17e1aa service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:34.386734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 747/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:39:36.882286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b862-8108-4df7-b7c4-0923d4bb9bd2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:36.882954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b862-8108-4df7-b7c4-0923d4bb9bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:36.882954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b862-8108-4df7-b7c4-0923d4bb9bd2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:36.884123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 744/1491] 10.222.0.73 () {54 vars in 926 bytes} [Thu May 12 17:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:39:36.889981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a7b49-41b1-40a5-a3d0-80caf2d3c17c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:36.890384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a7b49-41b1-40a5-a3d0-80caf2d3c17c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:36.890543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a7b49-41b1-40a5-a3d0-80caf2d3c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:36.908462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7d5a7b49-41b1-40a5-a3d0-80caf2d3c17c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:39:36.909640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7d5a7b49-41b1-40a5-a3d0-80caf2d3c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:39:36.917315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 748/1492] 10.222.0.73 () {58 vars in 1023 bytes} [Thu May 12 17:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:39:36.954824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b862-8108-4df7-b7c4-0923d4bb9bd2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:39:36.965849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b862-8108-4df7-b7c4-0923d4bb9bd2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:39:36.969274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1da158-d207-4bc5-8689-e8fae263009f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:39:36.969898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1da158-d207-4bc5-8689-e8fae263009f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:39:36.970178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1da158-d207-4bc5-8689-e8fae263009f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:39:36.970449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1da158-d207-4bc5-8689-e8fae263009f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:39:36.979013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1da158-d207-4bc5-8689-e8fae263009f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:39:36.980453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1da158-d207-4bc5-8689-e8fae263009f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:39:36.989587 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 745/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:41:04.701369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:04.701959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:04.701959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:04.720244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:41:04.721420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:04.729276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 749/1494] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:41:04.738539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:04.739028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:04.739124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:04.756596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:41:04.757773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:04.765946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 746/1495] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:41:04.770292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:04.781572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dbda4-c2bf-4733-a214-a7cfb76c63b5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:04.786029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:04.786679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:04.786968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:04.787241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:04.787964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:41:04.788304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:04.789304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:04.789304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:41:04.789304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:41:04.790174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:04.801534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 750/1496] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:41:04.809747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:04.818534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4c59-8e82-41e6-9c06-4a4fb225361a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:04.823647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:04.824306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:04.824584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:04.824850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:04.825526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:04.827241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:04.827241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:41:04.827241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:41:04.828428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:04.862924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 747/1497] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:41:04 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:41:04.867114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:04.876925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b9e73-2bc7-45ae-abcd-98998fc16d77 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:04.930194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:04.930850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:04.931677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:04.931952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:04.932686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:04.933686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:04.933686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:41:04.933686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:41:04.934883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:04.954935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 751/1498] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:41:04 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:41:04.959006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:04.968907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380ba8-129d-4cb0-b5b5-b28dfb17f659 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:04.973293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:04.973933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:04.974209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:04.974477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:04.975190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:04.976210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:04.976210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:41:04.976210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:41:04.977077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:04.989742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 748/1499] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:41:04.994261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:05.004411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670533b9-3396-42d9-9d13-5c686d232af2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:05.008913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:05.010318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.010597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.010870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.011605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:05.012637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:05.012637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:41:05.012637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:41:05.013523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7812f-30bb-4621-bc4b-daafff5f9993 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:05.027582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 752/1500] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:41:05.031839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:05.041431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f742ea-11d7-4add-a239-1199ef6e5fa5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:05.045868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:05.046733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.047014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.047288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] PATH_INFO: `/v3/projects/7e3cbe5ffece4b4294ec95e21d8c2294/users/6861b6f8d55c45d0948938b1f20f21d3/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.062341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3cbe5ffece4b4294ec95e21d8c2294, user_id=6861b6f8d55c45d0948938b1f20f21d3, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:05.063356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:05.063356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:41:05.063356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:41:05.065321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a03416-c5d9-4e1c-ad78-de447ed72bdd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:05.078320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 749/1501] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:41:05 2022] PUT /identity/v3/projects/7e3cbe5ffece4b4294ec95e21d8c2294/users/6861b6f8d55c45d0948938b1f20f21d3/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:41:05.089881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818113-a88d-4694-b016-7fbc7517a95e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.090174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818113-a88d-4694-b016-7fbc7517a95e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.090451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818113-a88d-4694-b016-7fbc7517a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.115414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b3818113-a88d-4694-b016-7fbc7517a95e None None] MFA Rules not processed for user `6861b6f8d55c45d0948938b1f20f21d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:41:05.116656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b3818113-a88d-4694-b016-7fbc7517a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:05.178910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 753/1502] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:41:05.192711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52553e89-f64d-477c-8a63-0a18ebaf363e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.193027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52553e89-f64d-477c-8a63-0a18ebaf363e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.193303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52553e89-f64d-477c-8a63-0a18ebaf363e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.194645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 750/1503] 10.222.0.73 () {54 vars in 963 bytes} [Thu May 12 17:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:41:05.200273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126919bf-9962-4c9f-ab40-1e9e3b3f6476 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.200560 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126919bf-9962-4c9f-ab40-1e9e3b3f6476 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.200859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126919bf-9962-4c9f-ab40-1e9e3b3f6476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.217881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-126919bf-9962-4c9f-ab40-1e9e3b3f6476 None None] MFA Rules not processed for user `71263b9cdb9d43ca9ce74c3e2069265f`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:41:05.219037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-126919bf-9962-4c9f-ab40-1e9e3b3f6476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:05.268398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 754/1504] 10.222.0.73 () {58 vars in 1060 bytes} [Thu May 12 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:41:05.275968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52553e89-f64d-477c-8a63-0a18ebaf363e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:05.286291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52553e89-f64d-477c-8a63-0a18ebaf363e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:05.289800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:05.290460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.290700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.290969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.298998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:05.300440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:05.309391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 751/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:41:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:41:05.351487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c551c-b629-483f-b79d-1ba4875a9735 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.351772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c551c-b629-483f-b79d-1ba4875a9735 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.352047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c551c-b629-483f-b79d-1ba4875a9735 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.353874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 755/1506] 10.222.0.73 () {52 vars in 829 bytes} [Thu May 12 17:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:41:05.357481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:05.416784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04197f96-b093-4da4-9f26-0b168747a7c1 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:05.421268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:05.422149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.422429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.422701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] PATH_INFO: `/v3/users/6861b6f8d55c45d0948938b1f20f21d3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.424334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6861b6f8d55c45d0948938b1f20f21d3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:05.425330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:05.425330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:41:05.425330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:41:05.426529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:05.490645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6861b6f8d55c45d0948938b1f20f21d3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:05.503425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6861b6f8d55c45d0948938b1f20f21d3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:05.513747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6861b6f8d55c45d0948938b1f20f21d3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:05.514421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6861b6f8d55c45d0948938b1f20f21d3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:05.532347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6861b6f8d55c45d0948938b1f20f21d3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:05.538510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 752/1507] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:41:05 2022] DELETE /identity/v3/users/6861b6f8d55c45d0948938b1f20f21d3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:41:05.543955 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.544262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.544534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.561766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:41:05.562929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:05.617934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 756/1508] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:41:05.628189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:05.629555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:05.684285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de2661-a5f2-4cf8-b33e-7288b8ce2d11 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:05.687726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaea41d-a964-445f-9c5e-190367774e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:05.688384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaea41d-a964-445f-9c5e-190367774e29 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:05.688664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaea41d-a964-445f-9c5e-190367774e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:05.688936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaea41d-a964-445f-9c5e-190367774e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:05.697900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaea41d-a964-445f-9c5e-190367774e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:05.699302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaea41d-a964-445f-9c5e-190367774e29 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:05.708256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 753/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:41:06.049129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:41:06.050562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:41:06.062854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f634f2-8447-4b24-9ddc-4021dc7b3fe7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:41:06.068192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:41:06.069156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:41:06.069447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:41:06.069859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] PATH_INFO: `/v3/projects/7e3cbe5ffece4b4294ec95e21d8c2294` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:41:06.071068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3cbe5ffece4b4294ec95e21d8c2294)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:41:06.073317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:41:06.073317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:41:06.073317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:41:06.074644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:41:06.136426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3cbe5ffece4b4294ec95e21d8c2294', 'request_id': 'req-707f0282-4941-4e5f-a801-d83fe0e8d687'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:06.145696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3cbe5ffece4b4294ec95e21d8c2294', 'request_id': 'req-707f0282-4941-4e5f-a801-d83fe0e8d687'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:06.151715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3cbe5ffece4b4294ec95e21d8c2294', 'request_id': 'req-707f0282-4941-4e5f-a801-d83fe0e8d687'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:06.170336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] The token cache is being invalidate because project 7e3cbe5ffece4b4294ec95e21d8c2294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:41:06.170626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:06.171267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-707f0282-4941-4e5f-a801-d83fe0e8d687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:41:06.172922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 757/1510] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:41:06 2022] DELETE /identity/v3/projects/7e3cbe5ffece4b4294ec95e21d8c2294 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:42:50.116601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52477f51-0be0-41f6-b2ab-d50632a6f893 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:50.117195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52477f51-0be0-41f6-b2ab-d50632a6f893 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:50.117195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52477f51-0be0-41f6-b2ab-d50632a6f893 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:50.118503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:42:50.123648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cf34f-6dfc-4048-bcc5-999b18dbe9bd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:50.124102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cf34f-6dfc-4048-bcc5-999b18dbe9bd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:50.124230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cf34f-6dfc-4048-bcc5-999b18dbe9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:50.141765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-235cf34f-6dfc-4048-bcc5-999b18dbe9bd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:42:50.142974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-235cf34f-6dfc-4048-bcc5-999b18dbe9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:42:50.198692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:42:50.368498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52477f51-0be0-41f6-b2ab-d50632a6f893 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:42:50.370121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-52477f51-0be0-41f6-b2ab-d50632a6f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:42:50.440617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52477f51-0be0-41f6-b2ab-d50632a6f893 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:42:50.445868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f629f8c-d7c2-406c-8326-12b871e07640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:42:50.446508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f629f8c-d7c2-406c-8326-12b871e07640 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:50.446790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f629f8c-d7c2-406c-8326-12b871e07640 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:50.447066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f629f8c-d7c2-406c-8326-12b871e07640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:50.455516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f629f8c-d7c2-406c-8326-12b871e07640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:42:50.457107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f629f8c-d7c2-406c-8326-12b871e07640 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:42:50.466785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 755/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:42:50.920972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db681-b25c-40bd-9a1a-62dfbb47af4a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:50.921260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db681-b25c-40bd-9a1a-62dfbb47af4a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:50.921533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db681-b25c-40bd-9a1a-62dfbb47af4a None None] PATH_INFO: `/v3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:50.922884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 759/1514] 10.222.0.74 () {54 vars in 925 bytes} [Thu May 12 17:42:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:42:50.928969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd89f5b-4b41-427a-a813-5c1073cd9311 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:50.929261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd89f5b-4b41-427a-a813-5c1073cd9311 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:50.929543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd89f5b-4b41-427a-a813-5c1073cd9311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:50.947830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3bd89f5b-4b41-427a-a813-5c1073cd9311 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:42:50.949032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3bd89f5b-4b41-427a-a813-5c1073cd9311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:42:51.011123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 756/1515] 10.222.0.74 () {58 vars in 1015 bytes} [Thu May 12 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:42:51.383175 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d6c5-ca01-4d43-b760-e50ff15c10b5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:51.383793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d6c5-ca01-4d43-b760-e50ff15c10b5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:51.383793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d6c5-ca01-4d43-b760-e50ff15c10b5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:51.385064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:42:51.390104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f6a15-ca01-4b0a-b361-116360d03985 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:51.390507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f6a15-ca01-4b0a-b361-116360d03985 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:51.390688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f6a15-ca01-4b0a-b361-116360d03985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:51.408672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-904f6a15-ca01-4b0a-b361-116360d03985 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:42:51.409881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-904f6a15-ca01-4b0a-b361-116360d03985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:42:51.417549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:42:51.570656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d6c5-ca01-4d43-b760-e50ff15c10b5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:42:51.585528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d6c5-ca01-4d43-b760-e50ff15c10b5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:42:51.591466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f0b36-0bae-4e79-ad0d-cad7c7a6e942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:42:51.592599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f0b36-0bae-4e79-ad0d-cad7c7a6e942 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:42:51.592919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f0b36-0bae-4e79-ad0d-cad7c7a6e942 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:42:51.593201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f0b36-0bae-4e79-ad0d-cad7c7a6e942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:42:51.601394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f0b36-0bae-4e79-ad0d-cad7c7a6e942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:42:51.602984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f0b36-0bae-4e79-ad0d-cad7c7a6e942 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:42:51.612713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 761/1518] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:00.071887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab25d50-4890-4c9a-9953-0e180322a0af None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:00.072488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab25d50-4890-4c9a-9953-0e180322a0af None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:00.072488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab25d50-4890-4c9a-9953-0e180322a0af None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:00.073812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 758/1519] 10.222.0.73 () {54 vars in 960 bytes} [Thu May 12 17:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:00.079163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dba6ae-6c17-4fa6-a8fe-fe1db80de0d3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:00.079575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dba6ae-6c17-4fa6-a8fe-fe1db80de0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:00.079713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dba6ae-6c17-4fa6-a8fe-fe1db80de0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:00.100115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-59dba6ae-6c17-4fa6-a8fe-fe1db80de0d3 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:00.101919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-59dba6ae-6c17-4fa6-a8fe-fe1db80de0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:00.113871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 762/1520] 10.222.0.73 () {58 vars in 1057 bytes} [Thu May 12 17:43:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:43:00.123063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab25d50-4890-4c9a-9953-0e180322a0af None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:00.136599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab25d50-4890-4c9a-9953-0e180322a0af None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:00.142051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05844b-425a-409b-8658-04c31324de2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:00.142690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05844b-425a-409b-8658-04c31324de2a service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:00.142975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05844b-425a-409b-8658-04c31324de2a service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:00.143251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05844b-425a-409b-8658-04c31324de2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:00.151275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05844b-425a-409b-8658-04c31324de2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:00.152899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05844b-425a-409b-8658-04c31324de2a service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:00.163087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 759/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:02.923311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3688-f9f4-45f6-af96-d52ec6a48d4a None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:02.923865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3688-f9f4-45f6-af96-d52ec6a48d4a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:02.923865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3688-f9f4-45f6-af96-d52ec6a48d4a None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:02.925294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:02.929983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72d39-c0f6-4211-96cb-1da843f472ec None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:02.930442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72d39-c0f6-4211-96cb-1da843f472ec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:02.930551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72d39-c0f6-4211-96cb-1da843f472ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:02.951090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2fc72d39-c0f6-4211-96cb-1da843f472ec None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:02.952341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2fc72d39-c0f6-4211-96cb-1da843f472ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:02.960067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:43:03.112289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3688-f9f4-45f6-af96-d52ec6a48d4a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:03.127196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3688-f9f4-45f6-af96-d52ec6a48d4a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:03.132746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ad4de-8dd0-4af0-98d7-c67c73868509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:03.133392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad4de-8dd0-4af0-98d7-c67c73868509 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:03.133676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad4de-8dd0-4af0-98d7-c67c73868509 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:03.133953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad4de-8dd0-4af0-98d7-c67c73868509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:03.143830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ad4de-8dd0-4af0-98d7-c67c73868509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:03.145507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ad4de-8dd0-4af0-98d7-c67c73868509 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:03.156217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 764/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:04.081919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd4f6b-ee86-463f-bf69-00004df076fa None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:04.082494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd4f6b-ee86-463f-bf69-00004df076fa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:04.082494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd4f6b-ee86-463f-bf69-00004df076fa None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:04.083834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 12 17:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:04.088667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9888fb-ae3c-47d1-9276-66b3b86ac44a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:04.089082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9888fb-ae3c-47d1-9276-66b3b86ac44a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:04.089227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9888fb-ae3c-47d1-9276-66b3b86ac44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:04.108467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7f9888fb-ae3c-47d1-9276-66b3b86ac44a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:04.109665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7f9888fb-ae3c-47d1-9276-66b3b86ac44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:04.117341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 12 17:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:43:04.269665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd4f6b-ee86-463f-bf69-00004df076fa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:04.284215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd4f6b-ee86-463f-bf69-00004df076fa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:04.289552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86eb99-117d-466a-b18c-aed763abcd59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:04.290183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86eb99-117d-466a-b18c-aed763abcd59 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:04.290463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86eb99-117d-466a-b18c-aed763abcd59 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:04.290734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86eb99-117d-466a-b18c-aed763abcd59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:04.298673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86eb99-117d-466a-b18c-aed763abcd59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:04.300262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86eb99-117d-466a-b18c-aed763abcd59 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:04.309287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 762/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:08.494042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56146a-6f1a-46df-91a6-8d47d9a983b5 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:08.494619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56146a-6f1a-46df-91a6-8d47d9a983b5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:08.494619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56146a-6f1a-46df-91a6-8d47d9a983b5 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:08.495941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:08.502247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0182-bb6d-48e5-80fa-6fa8884f44ea None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:08.502684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0182-bb6d-48e5-80fa-6fa8884f44ea None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:08.502789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0182-bb6d-48e5-80fa-6fa8884f44ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:08.520320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1f4f0182-bb6d-48e5-80fa-6fa8884f44ea None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:08.521493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f4f0182-bb6d-48e5-80fa-6fa8884f44ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:08.529758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:43:08.566507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56146a-6f1a-46df-91a6-8d47d9a983b5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:08.568205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ce56146a-6f1a-46df-91a6-8d47d9a983b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:08.626429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56146a-6f1a-46df-91a6-8d47d9a983b5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:08.629954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8f68f-52be-46b7-98d9-ca79767dc2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:08.630598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f68f-52be-46b7-98d9-ca79767dc2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:08.630881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f68f-52be-46b7-98d9-ca79767dc2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:08.631180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f68f-52be-46b7-98d9-ca79767dc2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:08.640298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8f68f-52be-46b7-98d9-ca79767dc2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:08.641725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8f68f-52be-46b7-98d9-ca79767dc2e2 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:08.650877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 767/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:10.854261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891502a-3c1b-4c2a-b5da-48078eaa3655 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:10.854869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891502a-3c1b-4c2a-b5da-48078eaa3655 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:10.854869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891502a-3c1b-4c2a-b5da-48078eaa3655 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:10.856182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:10.862420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f146762-651a-43a6-861d-b2f550be244c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:10.862875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f146762-651a-43a6-861d-b2f550be244c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:10.862991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f146762-651a-43a6-861d-b2f550be244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:10.881523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1f146762-651a-43a6-861d-b2f550be244c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:10.882639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1f146762-651a-43a6-861d-b2f550be244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:10.890651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:43:10.926355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891502a-3c1b-4c2a-b5da-48078eaa3655 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:10.936794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891502a-3c1b-4c2a-b5da-48078eaa3655 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:10.940232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e366e-c62a-4b46-99d8-a2177a9f6646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:10.940856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e366e-c62a-4b46-99d8-a2177a9f6646 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:10.941136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e366e-c62a-4b46-99d8-a2177a9f6646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:10.941407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e366e-c62a-4b46-99d8-a2177a9f6646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:10.949925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e366e-c62a-4b46-99d8-a2177a9f6646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:10.951504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e366e-c62a-4b46-99d8-a2177a9f6646 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:10.960535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 765/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:14.919760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915d95f-94f1-4731-957b-cef97c275ccc None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:14.920368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915d95f-94f1-4731-957b-cef97c275ccc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:14.920556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915d95f-94f1-4731-957b-cef97c275ccc None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:14.921977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:14.927324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786efe24-666d-47af-bc92-a2e2e75340c0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:14.927738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786efe24-666d-47af-bc92-a2e2e75340c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:14.927854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786efe24-666d-47af-bc92-a2e2e75340c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:14.945173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-786efe24-666d-47af-bc92-a2e2e75340c0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:14.946345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-786efe24-666d-47af-bc92-a2e2e75340c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:14.954872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:43:15.227315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915d95f-94f1-4731-957b-cef97c275ccc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:15.229003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e915d95f-94f1-4731-957b-cef97c275ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:15.283797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915d95f-94f1-4731-957b-cef97c275ccc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:15.288319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30aa26d-d123-4a71-b39b-6a2f9d06a171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:15.289075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aa26d-d123-4a71-b39b-6a2f9d06a171 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:15.289363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aa26d-d123-4a71-b39b-6a2f9d06a171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:15.289641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aa26d-d123-4a71-b39b-6a2f9d06a171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:15.298609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30aa26d-d123-4a71-b39b-6a2f9d06a171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:15.300039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30aa26d-d123-4a71-b39b-6a2f9d06a171 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:15.309193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 770/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:43:17.583383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd44e29-a541-4596-aa70-eb823d42981b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:17.583949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd44e29-a541-4596-aa70-eb823d42981b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:17.583949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd44e29-a541-4596-aa70-eb823d42981b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:17.585316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 12 17:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:43:17.590692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881829c-2895-48c7-97a0-a0c0fb08289a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:17.591102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881829c-2895-48c7-97a0-a0c0fb08289a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:17.591283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881829c-2895-48c7-97a0-a0c0fb08289a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:17.609935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9881829c-2895-48c7-97a0-a0c0fb08289a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:43:17.611155 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9881829c-2895-48c7-97a0-a0c0fb08289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:17.618917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 12 17:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:43:17.888398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd44e29-a541-4596-aa70-eb823d42981b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:43:17.890680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3cd44e29-a541-4596-aa70-eb823d42981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:43:17.903192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd44e29-a541-4596-aa70-eb823d42981b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:43:17.907277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d920a0-8147-4e3b-bcca-7157ca9a8920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:43:17.907905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d920a0-8147-4e3b-bcca-7157ca9a8920 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:43:17.908204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d920a0-8147-4e3b-bcca-7157ca9a8920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:43:17.908482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d920a0-8147-4e3b-bcca-7157ca9a8920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:43:17.917010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d920a0-8147-4e3b-bcca-7157ca9a8920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:43:17.918422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d920a0-8147-4e3b-bcca-7157ca9a8920 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:43:17.927764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 768/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:50.414292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cccf7-8958-4e7a-9727-765275e2c50f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.414943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cccf7-8958-4e7a-9727-765275e2c50f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.414943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cccf7-8958-4e7a-9727-765275e2c50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.432557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826a0dc-26c2-4948-9819-f98453eac923 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.433154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826a0dc-26c2-4948-9819-f98453eac923 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.433154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826a0dc-26c2-4948-9819-f98453eac923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.442594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a55cccf7-8958-4e7a-9727-765275e2c50f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.443801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a55cccf7-8958-4e7a-9727-765275e2c50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.458795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8826a0dc-26c2-4948-9819-f98453eac923 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.460006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8826a0dc-26c2-4948-9819-f98453eac923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.461985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 772/1540] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.473511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 769/1541] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:44:50.482555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23248f-3c3e-4490-aa38-40e019ea4700 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.482919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23248f-3c3e-4490-aa38-40e019ea4700 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.483788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23248f-3c3e-4490-aa38-40e019ea4700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.483998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fabf-240e-4f85-ad17-ec42c14db47b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.484321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fabf-240e-4f85-ad17-ec42c14db47b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.484599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fabf-240e-4f85-ad17-ec42c14db47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.513838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bae1fabf-240e-4f85-ad17-ec42c14db47b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.515139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bae1fabf-240e-4f85-ad17-ec42c14db47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.524321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 770/1542] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:44:50.525260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6d23248f-3c3e-4490-aa38-40e019ea4700 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.526003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d62744-b79f-4560-be42-34470f79da94 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.526446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d62744-b79f-4560-be42-34470f79da94 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.526578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6d23248f-3c3e-4490-aa38-40e019ea4700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.526985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d62744-b79f-4560-be42-34470f79da94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.549069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 773/1543] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.550456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.550754 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.551037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.552824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d6d62744-b79f-4560-be42-34470f79da94 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.554199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6d62744-b79f-4560-be42-34470f79da94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.563879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 771/1544] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.565644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faca7c-aa6b-4e70-8de4-f90af4491d76 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.566197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faca7c-aa6b-4e70-8de4-f90af4491d76 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.566676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faca7c-aa6b-4e70-8de4-f90af4491d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.579771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.581329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.592791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 774/1545] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.593971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.597845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f5faca7c-aa6b-4e70-8de4-f90af4491d76 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.599516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5faca7c-aa6b-4e70-8de4-f90af4491d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.605068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bb1d5-a461-4bdb-876d-376d5dc0db0c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.610605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.611312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 772/1546] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.613094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.613623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.614103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.615109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.615447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.615839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.616634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:50.616946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.620342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62370fff-bd66-48ed-9ce6-ebf9c61e8b58 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.639250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 775/1547] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:50.640745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934aa01-210e-4be8-a6c1-9f9b92400bdb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.641038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934aa01-210e-4be8-a6c1-9f9b92400bdb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.641330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934aa01-210e-4be8-a6c1-9f9b92400bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.645468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.646886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.656849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 773/1548] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.656849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.658784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5934aa01-210e-4be8-a6c1-9f9b92400bdb None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.659947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5934aa01-210e-4be8-a6c1-9f9b92400bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.665606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755205ca-a535-4bc5-875e-152cec4b94a6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.667793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 776/1549] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.669031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5b4d7-1b9a-4ea7-af43-6e0386558e5b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.669320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5b4d7-1b9a-4ea7-af43-6e0386558e5b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.669600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5b4d7-1b9a-4ea7-af43-6e0386558e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.670056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.671722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.671951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.672260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.672997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:50.673303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.674430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 12 17:44:50.674430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:44:50.674430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:44:50.675305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467f2b2-90f0-4fc4-a107-dd17672506e8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.687793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 774/1550] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:50.689093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09859c-09b4-442e-a1fc-8f6dc71656e4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.689384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09859c-09b4-442e-a1fc-8f6dc71656e4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.689660 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09859c-09b4-442e-a1fc-8f6dc71656e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.693212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-67a5b4d7-1b9a-4ea7-af43-6e0386558e5b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.694434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-67a5b4d7-1b9a-4ea7-af43-6e0386558e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.702343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 777/1551] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.703554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.703843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.704117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.711106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7e09859c-09b4-442e-a1fc-8f6dc71656e4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.712331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e09859c-09b4-442e-a1fc-8f6dc71656e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.720253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 775/1552] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.721538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.721664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.721827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.722104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.722726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.730581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 778/1553] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.739579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.740589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.741303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b7eb1-9ab5-4be4-a83c-01de735ce2d5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.741779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.745743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.746391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.746814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.747099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.747788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.749982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:44:50.749982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:44:50.749982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:44:50.750351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 776/1554] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.751179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8030ef-a3d1-432c-a7f5-9a00672d09f5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.751301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.760363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd690-0bc1-47f7-90a6-26e511b6e14d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.764993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.765653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.765974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.766265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.767054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:50.767389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.769125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.776559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 779/1555] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:50.777928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.778056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.778301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.785141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 777/1556] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:50.785141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.794233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9b4ca-d0b2-47d6-b206-cadce4b57d4d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.800392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.801578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.847794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.848476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.848755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.849032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.849705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.852155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.857430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 780/1557] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.858854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.868340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42aaac-1132-4e21-b114-ccd5bfea1f72 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.874364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.875121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.875398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.875784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.876552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:50.876931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.878648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e61d6-54a7-4a9f-90a2-c99574580d98 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.879989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 778/1558] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:50.881540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.898932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 781/1559] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:50.898932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504994ed-7472-4c33-93ce-b5077648f964 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.898932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504994ed-7472-4c33-93ce-b5077648f964 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.898932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504994ed-7472-4c33-93ce-b5077648f964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.912599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-504994ed-7472-4c33-93ce-b5077648f964 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.913778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-504994ed-7472-4c33-93ce-b5077648f964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.951160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f0bd9-a353-4d3f-a234-b0ba57e4a1f9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.956018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.956776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.957059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.957331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.958043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:50.958347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.959933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:50.970998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 782/1560] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:50.972392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.975260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.975409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 779/1561] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:50.975526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.975632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:50.986669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4419d66-be1b-418f-a66b-0c0737d0afd0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:50.991341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:50.992000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:50.992321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:50.992599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:50.993288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:50.993450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:50.994299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:44:50.994299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:44:50.994299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:44:50.994643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:50.995499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.002860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 783/1562] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:51.003609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.013564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86125677-fe2f-4974-a9d3-d94fda2bb3f2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.016538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 780/1563] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:50 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.017246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.021304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.021958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.022243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.022520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.023201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.025132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.028420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7d7a7-14c1-4323-9d9f-74e4c41efcd2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.033400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.034016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.034302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.034579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.035284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:51.035589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.037181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.050725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 781/1564] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:51.051674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.055826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 784/1565] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.055826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.063486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e3e24-9436-4ba5-8b50-31cba244cdd7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.115715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc23c-1966-417a-8609-cbf2a63943a2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.116297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.117060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.117372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.117644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.118323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.120250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.120408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.121030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.121313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.121585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.122260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.125172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.141103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 782/1566] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.141914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.151422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e6a2f-2ab5-4986-a685-886f7a30b1a9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.155956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.156628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.156909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.157179 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.157670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 785/1567] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.157905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:51.158276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.158380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.159866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.172279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 783/1568] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:51.173350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.184467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca2924-18f7-44ff-b572-70b887fd6a5f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.219395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e233c-d572-4ab1-9450-a2d9ed2430d2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.223865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.224558 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.224846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.225122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.225805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.227705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.238850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.239503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.239784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.240058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.240806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:44:51.241110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.242808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.256673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 786/1569] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.256673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.256987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 784/1570] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:44:51.256987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.267808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3034e-8b77-47ad-9d7a-6cb216cba622 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.319556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078bb9e-c2d4-40d4-b92d-e46e885d5b45 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.324130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.324821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.325107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.325383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.325603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.326114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.326263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.327076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.327679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.328018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.328642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.329936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.338312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 787/1571] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.339163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.349555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d80a5-6e3b-4801-a67a-cc2ee54eeac6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.351513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 785/1572] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.352261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.354427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.355085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.355370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.355648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.356363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.358290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.361686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c307-8e9e-4d7b-a894-8922b5cc3802 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.366075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.366709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.367067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.367346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.368055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.369627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.381377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 786/1573] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.382155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.387793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 788/1574] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.388739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.395545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a7f39-6f57-4bc5-ac3b-67eb3e5847ae admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.448525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8397d-3943-44eb-bca8-8990cc0d9428 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.449666 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.450332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.450644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.450918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.451601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.453283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.453566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.453926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.454211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.454480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.455151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.457067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.474813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 787/1575] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.475689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.484478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 789/1576] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.485142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.490690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215aa28-dcbd-407e-a976-261198247a8f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.544519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.544736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80580-b811-42a7-b9ec-1d1cfa21b033 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.545183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.546109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.546387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.547098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.549053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.549763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.550439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.550714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.550986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.551656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.553538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.569430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 788/1577] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.570262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.579975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c6b03-7752-48b2-9a33-c97b7bd3e422 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.581514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 790/1578] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.581652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.643312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.643995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.644323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.644600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.653298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.656311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.667041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 789/1579] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.667918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.679426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccfa3d-4721-491b-8b78-2f81295ffade admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.684510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.685194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.685484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.685763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.687548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.689216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.700764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 790/1580] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.701668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.714056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478b0d4-2d0b-44e4-9729-a2f5d1be3a8a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.719750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.720527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.720840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.721111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.721835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.723826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.744679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 791/1581] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.745571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.755317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053de8-7f30-423c-a79c-fd9b33bd037c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.760148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.760824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.761109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.761387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.762103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.763682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.773444 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662a8e8-d5e8-4f44-9416-dbbccf11f174 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.773796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 792/1582] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.778186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.778352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.778850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.779128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.779395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.780817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.782549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.788794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7013d-cfab-4b2b-b00e-450dad481e47 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.792923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 791/1583] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.793387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.793832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.794205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.794483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.794760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.795506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.798152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.807719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a302ca-65f0-4191-a0c0-6d9020ba8b6d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.813451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.814105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.814381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.814682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.815417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.817018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.817885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 793/1584] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.818758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.827984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 792/1585] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.828827 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.831046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587a56-e6b3-4190-96ff-4aeaf56a2ef1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.835617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.836325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.836613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.836924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.837603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.839361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04770b-cc1e-4d5f-805e-36bee517b9da admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.840550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.844925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.845784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.846057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.846322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] PATH_INFO: `/v3/projects/a3227d2362fb4a33ac4288176db85394/users/cae2f818c44f4f70b38e0dda4b189bae/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.861897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3227d2362fb4a33ac4288176db85394, user_id=cae2f818c44f4f70b38e0dda4b189bae, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.862420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 794/1586] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:51 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:51.863007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:44:51.863007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:44:51.863007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:44:51.863271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.865029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.873345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488ec77-0a10-492a-abbe-389ac6bc4631 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.879322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 793/1587] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:51 2022] PUT /identity/v3/projects/a3227d2362fb4a33ac4288176db85394/users/cae2f818c44f4f70b38e0dda4b189bae/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:51.880228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.930763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.931647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.932034 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.932340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] PATH_INFO: `/v3/projects/53cc45c6ac3e48a7aa2041ffd965d3db/users/57026d850c044b578bcb991e03b76aee/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.939319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc93574-308e-4a3d-a156-3dfb9a8cd57b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:51.943859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:51.944547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:51.944826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:51.945099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:51.945820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.946374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53cc45c6ac3e48a7aa2041ffd965d3db, user_id=57026d850c044b578bcb991e03b76aee, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:51.947404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.949410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:51.957109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 794/1588] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:51.957953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.962524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 795/1589] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:51 2022] PUT /identity/v3/projects/53cc45c6ac3e48a7aa2041ffd965d3db/users/57026d850c044b578bcb991e03b76aee/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:51.963419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:51.967761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a799e-d421-4f11-8dc9-f6d91d890e32 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.021892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.022472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.022749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.023064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.023784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.023907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33439346-5578-4e12-b3a1-853136ac7133 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.025408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.028530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.029177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.029453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.030285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.031007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.032600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.035630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 795/1590] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.036789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.045299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 796/1591] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.045299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.046439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb9d9c-8765-4539-8cd0-c8f92b8c4b89 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.051242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.051914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.052210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.052483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.053200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.054699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e8de9-a9cf-465e-a319-d6ab228bc9fb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.054796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.059350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c3be-4f8a-41d2-821f-9b069f62cea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.059986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c3be-4f8a-41d2-821f-9b069f62cea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.060302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c3be-4f8a-41d2-821f-9b069f62cea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.060581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c3be-4f8a-41d2-821f-9b069f62cea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.061422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c3be-4f8a-41d2-821f-9b069f62cea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.063026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c3be-4f8a-41d2-821f-9b069f62cea3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.064557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 796/1592] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.065338 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.073824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 797/1593] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.074506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3cb3-9c14-483f-91a6-ce024bca6a27 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.075266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.075619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.075978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.079929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.080616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.080902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.081181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.081910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.083489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.094050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 797/1594] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.095561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.102677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] MFA Rules not processed for user `cae2f818c44f4f70b38e0dda4b189bae`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.103943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.105109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f388211-9ae4-4682-b228-70632775b389 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.109658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d38af3-e30f-4078-abab-e1d3461fd2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.110298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38af3-e30f-4078-abab-e1d3461fd2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.110579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38af3-e30f-4078-abab-e1d3461fd2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.110853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d38af3-e30f-4078-abab-e1d3461fd2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.111574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d38af3-e30f-4078-abab-e1d3461fd2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.113166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d38af3-e30f-4078-abab-e1d3461fd2e0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.126512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 798/1595] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.126512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.126512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.126512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.152094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] MFA Rules not processed for user `57026d850c044b578bcb991e03b76aee`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.153336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.167167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 798/1596] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.167979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.178499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479fcb1-edc8-4ae4-b5bd-4ca6e219401c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.185212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.186098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.186361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.186632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] PATH_INFO: `/v3/projects/627ccbef6d014765a87060bc4fb7bde3/users/8bd19c47ef9944c3b1791c61d513a5a2/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.203803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627ccbef6d014765a87060bc4fb7bde3, user_id=8bd19c47ef9944c3b1791c61d513a5a2, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.206432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.221068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 799/1597] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:52 2022] PUT /identity/v3/projects/627ccbef6d014765a87060bc4fb7bde3/users/8bd19c47ef9944c3b1791c61d513a5a2/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:52.222638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.231099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 799/1598] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.231821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.242070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a90682-b937-4c0b-afd3-f28c2982915c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.292757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ad8ac-35fb-4dbe-a14d-7ed88dba3423 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.297386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.298258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.298529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.298793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] PATH_INFO: `/v3/projects/bdb0d817bd8e4eeca5ed6229a4b5c53a/users/7a2b51125e9645dc9454042f49ff3659/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.299785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.300588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.300873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.301144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.301897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.303629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.313018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb0d817bd8e4eeca5ed6229a4b5c53a, user_id=7a2b51125e9645dc9454042f49ff3659, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.313179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 800/1599] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.316749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.316849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.326139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396210a-6917-49aa-ba20-a0b075cec60e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.329279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 800/1600] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:52 2022] PUT /identity/v3/projects/bdb0d817bd8e4eeca5ed6229a4b5c53a/users/7a2b51125e9645dc9454042f49ff3659/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:52.330078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.383146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.384029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.384355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.384634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] PATH_INFO: `/v3/projects/4a12890be016462ea430af4ff0080553/users/b00b6242e52a4ab1bc98280fe7242647/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.390765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72f4d6-4a0b-4dc3-8e20-ac101fadc2f2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.395385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.396042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.396361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.396640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.397349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.398823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a12890be016462ea430af4ff0080553, user_id=b00b6242e52a4ab1bc98280fe7242647, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.398934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.401510 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.409876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 801/1601] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:52.413318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.415148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 801/1602] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:52 2022] PUT /identity/v3/projects/4a12890be016462ea430af4ff0080553/users/b00b6242e52a4ab1bc98280fe7242647/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:52.416988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.423359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e7159-e4ea-45b5-9bb1-8c0c8b2efbc7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.476397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2cabb-56d6-42f7-8844-b1b8c5621354 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.477255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13200d4d-dda6-4430-a324-d4b68a8be17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.478140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13200d4d-dda6-4430-a324-d4b68a8be17e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.478422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13200d4d-dda6-4430-a324-d4b68a8be17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.478699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13200d4d-dda6-4430-a324-d4b68a8be17e admin admin] PATH_INFO: `/v3/projects/202db80dae2446e2b9cd1124b2925ced/users/298a13924a864f61adf6e4085e569358/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.481474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2814f-d599-4ab0-a562-f4346b6651fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.482203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2814f-d599-4ab0-a562-f4346b6651fd admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.482484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2814f-d599-4ab0-a562-f4346b6651fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.482783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2814f-d599-4ab0-a562-f4346b6651fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.483592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2814f-d599-4ab0-a562-f4346b6651fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.485534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2814f-d599-4ab0-a562-f4346b6651fd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.493285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13200d4d-dda6-4430-a324-d4b68a8be17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=202db80dae2446e2b9cd1124b2925ced, user_id=298a13924a864f61adf6e4085e569358, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.495935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13200d4d-dda6-4430-a324-d4b68a8be17e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.510279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 802/1603] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:52 2022] PUT /identity/v3/projects/202db80dae2446e2b9cd1124b2925ced/users/298a13924a864f61adf6e4085e569358/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:52.513921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 802/1604] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:52.514662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a677a7-c96a-4930-8b2b-6d60e4f5e2c3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.514662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a677a7-c96a-4930-8b2b-6d60e4f5e2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.514936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053080-4cd7-4690-83fa-666410b324ec None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.515026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a677a7-c96a-4930-8b2b-6d60e4f5e2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.515115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053080-4cd7-4690-83fa-666410b324ec None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.515115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053080-4cd7-4690-83fa-666410b324ec None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.515115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 803/1605] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 12 17:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:44:52.515916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5053080-4cd7-4690-83fa-666410b324ec None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.541002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b1a677a7-c96a-4930-8b2b-6d60e4f5e2c3 None None] MFA Rules not processed for user `8bd19c47ef9944c3b1791c61d513a5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.542378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1a677a7-c96a-4930-8b2b-6d60e4f5e2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.579645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5053080-4cd7-4690-83fa-666410b324ec None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.586340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.587232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.587509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.587781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] PATH_INFO: `/v3/projects/0eee0523f68a4d8dbc6db8de3a3f6b04/users/1cc16878bc6a445f8a66dfcdc54a7e7b/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.604108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eee0523f68a4d8dbc6db8de3a3f6b04, user_id=1cc16878bc6a445f8a66dfcdc54a7e7b, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.606797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.612667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 803/1606] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.614184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72f2ce-1018-42cb-b071-c3d9a3f7e639 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.614475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72f2ce-1018-42cb-b071-c3d9a3f7e639 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.614833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72f2ce-1018-42cb-b071-c3d9a3f7e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.626293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 804/1607] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:52 2022] PUT /identity/v3/projects/0eee0523f68a4d8dbc6db8de3a3f6b04/users/1cc16878bc6a445f8a66dfcdc54a7e7b/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:52.627189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.636180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ae72f2ce-1018-42cb-b071-c3d9a3f7e639 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.637463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ae72f2ce-1018-42cb-b071-c3d9a3f7e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.686964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b87327-017a-40fe-9fd6-3c886652f83e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.691618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.692566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.692841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.693108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] PATH_INFO: `/v3/projects/c40177cf7be6471da409b9698dd5db5a/users/d3c959294173455f86249100dbcf6d07/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.697276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 804/1608] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.698635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2d78-b7f3-48e0-a453-41e18bab064d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.698929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2d78-b7f3-48e0-a453-41e18bab064d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.699216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2d78-b7f3-48e0-a453-41e18bab064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.707568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40177cf7be6471da409b9698dd5db5a, user_id=d3c959294173455f86249100dbcf6d07, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.710303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.721387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-568a2d78-b7f3-48e0-a453-41e18bab064d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.722589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-568a2d78-b7f3-48e0-a453-41e18bab064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.724158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 805/1609] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:52 2022] PUT /identity/v3/projects/c40177cf7be6471da409b9698dd5db5a/users/d3c959294173455f86249100dbcf6d07/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:52.735284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.779191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 805/1610] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.781087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d87fe5-1c5f-41af-9ac8-ae4f6b40085e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.781375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d87fe5-1c5f-41af-9ac8-ae4f6b40085e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.781660 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d87fe5-1c5f-41af-9ac8-ae4f6b40085e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.794105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd8df8-2baf-4049-b1e4-de00d5c2953d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.799125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-44d87fe5-1c5f-41af-9ac8-ae4f6b40085e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.799285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.799894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.800199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.800436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-44d87fe5-1c5f-41af-9ac8-ae4f6b40085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.800596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.801139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.803029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.807836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 806/1611] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.809058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.809449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.809611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.824843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 806/1612] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:52.825644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.831975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.833143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.836470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cbb6f-e1fe-4663-a67f-4f65f76dab06 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.841128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d9c8c-c4bb-450e-9bde-dbf811074217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:52.841764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d9c8c-c4bb-450e-9bde-dbf811074217 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.842047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d9c8c-c4bb-450e-9bde-dbf811074217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.842940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d9c8c-c4bb-450e-9bde-dbf811074217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.843746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d9c8c-c4bb-450e-9bde-dbf811074217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:52.845668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d9c8c-c4bb-450e-9bde-dbf811074217 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:52.870619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 807/1613] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:52.871956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac9830-19a4-4734-bd74-482cde139732 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.872336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac9830-19a4-4734-bd74-482cde139732 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.872566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac9830-19a4-4734-bd74-482cde139732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.891094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-73ac9830-19a4-4734-bd74-482cde139732 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.892344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-73ac9830-19a4-4734-bd74-482cde139732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.899994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 807/1614] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.900858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:52.902118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:52.953448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 808/1615] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:52.954828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:52.955123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:52.955395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:52.958452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17174e96-a41e-4a9b-a539-8881285724fc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:52.984490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] MFA Rules not processed for user `d3c959294173455f86249100dbcf6d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:52.985708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:53.006310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.006964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.007252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.007531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.017158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.018606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.029023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 808/1616] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:53.030518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.038983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246896b-5efd-4cca-a1c3-2efcacb5f4fd service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.042824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.043588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.043979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.044397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.054626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.056268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 809/1617] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:53.057107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.060039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.066836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa26490-039b-4dcc-8770-f13024ede076 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.070050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 809/1618] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:53.070867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.072582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.073478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.073912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.074471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.080713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d743a5-3689-4f03-8bf1-6e8db8676987 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.085294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.085725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.086001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.086272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.086544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.087152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.087260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.088838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.097832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 810/1619] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:53.098791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.101893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 810/1620] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:53.105586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.108349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acee43-c207-4554-929e-b10ac9a2f238 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.113836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.114183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.114514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.114787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.115494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.117417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.147263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 811/1621] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:53 2022] POST /identity/v3/users => generated 299 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:53.148781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.168023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cbe4b-97bb-4d58-ace5-e3f29f0b2d62 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.173399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.174094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.174380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.174652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.184833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.186782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.197117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 812/1622] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:53.197749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.199513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce6e57-a052-4ae6-aa53-3184e3be355c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.206662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.207585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aabba-517b-4a70-9dcd-cf4ccd41bb50 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.207788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.208408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.208862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.212350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.212993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.213306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.213574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.214245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.216409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.242707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 813/1623] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:44:53 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:53.243590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.278255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.280800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.291275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 811/1624] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:53.303403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.317140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d68a6-fe37-430c-9f3c-733388f014ff service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.318501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f30b5a-c1c2-4659-b734-379d3655ea0c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.323518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.324523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.324670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.324983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.325428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.325772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.326114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.326231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.326591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] PATH_INFO: `/v3/projects/2893bb227b564e909366a36266d9300e/users/e5ff273123a04779bf536978b7c58623/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.329088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.344113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2893bb227b564e909366a36266d9300e, user_id=e5ff273123a04779bf536978b7c58623, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.347066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 812/1625] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:53.347221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.348015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.360543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b3eab-fe26-4e1d-a7ba-678b570240ee admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.368476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 814/1626] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:53 2022] PUT /identity/v3/projects/2893bb227b564e909366a36266d9300e/users/e5ff273123a04779bf536978b7c58623/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:53.370279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.432998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.433674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.433954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.434278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.435698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.438840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.450380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cba56-2228-4b1e-bd97-e565343556a8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.456342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.457641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.457999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.458458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] PATH_INFO: `/v3/projects/8ca3768c1dcd44488f8859a4d4f1fd7c/users/585c74ff6d7246cf8b4bbe099c3ba986/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.464707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 813/1627] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:44:53 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:44:53.465862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.476101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca3768c1dcd44488f8859a4d4f1fd7c, user_id=585c74ff6d7246cf8b4bbe099c3ba986, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.478406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e972bb-ec31-4a0a-b204-0a3fb41001bd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.479399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.484428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.485132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.485410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.485680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.486405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.488209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.495555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 815/1628] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:53 2022] PUT /identity/v3/projects/8ca3768c1dcd44488f8859a4d4f1fd7c/users/585c74ff6d7246cf8b4bbe099c3ba986/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:53.496920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.504897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 814/1629] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:53.505603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.518397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8fdf0e-510b-4d9a-8b5a-80a24f769f9a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.576448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64523a52-b676-4e54-ba96-6c68b73afb73 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.583855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3188-a487-4423-a185-e1fdd6ba1e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.586123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.586764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3188-a487-4423-a185-e1fdd6ba1e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.586875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.587059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3188-a487-4423-a185-e1fdd6ba1e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.587161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.587339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3188-a487-4423-a185-e1fdd6ba1e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.587438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.588688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.590321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.596764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b3188-a487-4423-a185-e1fdd6ba1e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.599421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b3188-a487-4423-a185-e1fdd6ba1e8b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.607350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 816/1630] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:53.610103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.621931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 815/1631] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:53.624886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efcddb-0024-408c-87b7-1186ac80bc35 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.625515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efcddb-0024-408c-87b7-1186ac80bc35 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.626536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efcddb-0024-408c-87b7-1186ac80bc35 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.628978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 816/1632] 10.222.0.73 () {54 vars in 912 bytes} [Thu May 12 17:44:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:44:53.630357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efcddb-0024-408c-87b7-1186ac80bc35 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.660144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2fe98-e5e2-4a3a-9f6d-ace99a65f6c0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.666328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efcddb-0024-408c-87b7-1186ac80bc35 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.670632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.672298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.672684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.676084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] PATH_INFO: `/v3/projects/2893bb227b564e909366a36266d9300e/users/e5ff273123a04779bf536978b7c58623/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.677895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d59528-304b-4e73-aeea-6463498d664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.678785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59528-304b-4e73-aeea-6463498d664b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.679066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59528-304b-4e73-aeea-6463498d664b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.679356 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d59528-304b-4e73-aeea-6463498d664b admin admin] PATH_INFO: `/v3/projects/ed8681c7f61c4dcdb751b4ded0fe8eec/users/ab2e527fe8174ffe8e2bc6e4c43fbb5a/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.680427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2893bb227b564e909366a36266d9300e, user_id=e5ff273123a04779bf536978b7c58623, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.683457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.700316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 817/1633] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:53 2022] PUT /identity/v3/projects/2893bb227b564e909366a36266d9300e/users/e5ff273123a04779bf536978b7c58623/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:53.701242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:53.716067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d59528-304b-4e73-aeea-6463498d664b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8681c7f61c4dcdb751b4ded0fe8eec, user_id=ab2e527fe8174ffe8e2bc6e4c43fbb5a, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.721115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d59528-304b-4e73-aeea-6463498d664b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.741705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 817/1634] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:53 2022] PUT /identity/v3/projects/ed8681c7f61c4dcdb751b4ded0fe8eec/users/ab2e527fe8174ffe8e2bc6e4c43fbb5a/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:53.743378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6842f-eaf6-4a9d-9144-cdeebec00f35 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.743672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6842f-eaf6-4a9d-9144-cdeebec00f35 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.744011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6842f-eaf6-4a9d-9144-cdeebec00f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.771864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9dd6842f-eaf6-4a9d-9144-cdeebec00f35 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:53.773656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9dd6842f-eaf6-4a9d-9144-cdeebec00f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:53.844386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e57c5-995c-493b-a71a-5335bd235da1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:53.881017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 818/1635] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:44:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:53.884402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.884848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.885135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.922617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:53.923640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:53.923986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:53.924521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] PATH_INFO: `/v3/projects/8ca3768c1dcd44488f8859a4d4f1fd7c/users/585c74ff6d7246cf8b4bbe099c3ba986/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:53.927303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca3768c1dcd44488f8859a4d4f1fd7c, user_id=585c74ff6d7246cf8b4bbe099c3ba986, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:53.930404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:53.935047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] MFA Rules not processed for user `e5ff273123a04779bf536978b7c58623`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:53.937150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:53.955776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 818/1636] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:53 2022] PUT /identity/v3/projects/8ca3768c1dcd44488f8859a4d4f1fd7c/users/585c74ff6d7246cf8b4bbe099c3ba986/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:53.969062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:54.055794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d4297-4283-4a05-899b-e858b8a21002 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:54.063142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4965eca-e1cd-4989-8bb1-d633650d955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:54.063872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4965eca-e1cd-4989-8bb1-d633650d955a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.064174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4965eca-e1cd-4989-8bb1-d633650d955a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.064566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4965eca-e1cd-4989-8bb1-d633650d955a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.065358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4965eca-e1cd-4989-8bb1-d633650d955a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:54.066822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 819/1637] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:54.067798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:54.072208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4965eca-e1cd-4989-8bb1-d633650d955a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:54.086981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 819/1638] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:44:54.088707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.089043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.089681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.127973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] MFA Rules not processed for user `585c74ff6d7246cf8b4bbe099c3ba986`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:54.129383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:54.198624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d53b6-bd6f-408d-90ca-f58d540c0319 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:54.267291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 820/1639] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:54.267291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:54.302353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:54.303389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.303916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.304583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.353952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94036b-ec3f-48a1-ac52-001ca62c8d97 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:54.359823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:54.360735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.361380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.361380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.411804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:54.413667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:54.427310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 820/1640] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:54.428322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:54.440012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5b394-50aa-4acb-af55-a85142819aef service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:54.446387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352cff-2b39-4b12-82c7-f1ee1d4badb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:54.447670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352cff-2b39-4b12-82c7-f1ee1d4badb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.448237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352cff-2b39-4b12-82c7-f1ee1d4badb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.448716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352cff-2b39-4b12-82c7-f1ee1d4badb6 admin admin] PATH_INFO: `/v3/projects/ed8681c7f61c4dcdb751b4ded0fe8eec/users/ab2e527fe8174ffe8e2bc6e4c43fbb5a/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.451412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352cff-2b39-4b12-82c7-f1ee1d4badb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8681c7f61c4dcdb751b4ded0fe8eec, user_id=ab2e527fe8174ffe8e2bc6e4c43fbb5a, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:54.457703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352cff-2b39-4b12-82c7-f1ee1d4badb6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:54.473485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 821/1641] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:44:54 2022] PUT /identity/v3/projects/ed8681c7f61c4dcdb751b4ded0fe8eec/users/ab2e527fe8174ffe8e2bc6e4c43fbb5a/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:54.490483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.491145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.491145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.503631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:54.505337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:54.517346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 821/1642] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:54.528215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] MFA Rules not processed for user `ab2e527fe8174ffe8e2bc6e4c43fbb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:54.529424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:54.603588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:54.617533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffc3a8-ccf0-4559-a57e-81cc2c8ca8f5 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:54.652222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 822/1643] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:54.715942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729c6d5-caa5-4bc9-9486-dfee0434134a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:54.717025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729c6d5-caa5-4bc9-9486-dfee0434134a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:54.718014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729c6d5-caa5-4bc9-9486-dfee0434134a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:54.718777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729c6d5-caa5-4bc9-9486-dfee0434134a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:54.813805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729c6d5-caa5-4bc9-9486-dfee0434134a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:54.816676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729c6d5-caa5-4bc9-9486-dfee0434134a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:54.843550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 822/1644] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:55.180877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:55.199967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99b83-f426-4908-b606-d0a1d373bee4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:55.203860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1e1e-b2f0-49eb-8212-efd80af6be39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:55.204538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1e1e-b2f0-49eb-8212-efd80af6be39 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.205357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1e1e-b2f0-49eb-8212-efd80af6be39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.205357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1e1e-b2f0-49eb-8212-efd80af6be39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.232150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1e1e-b2f0-49eb-8212-efd80af6be39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:55.234769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1e1e-b2f0-49eb-8212-efd80af6be39 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:55.254892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 823/1645] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:55.261797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.262359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.262838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.338513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] MFA Rules not processed for user `b00b6242e52a4ab1bc98280fe7242647`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:55.339851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:55.494192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 823/1646] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:55.498543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473213-2349-48f0-8cb8-3713b602fdfe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.499392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473213-2349-48f0-8cb8-3713b602fdfe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.499392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473213-2349-48f0-8cb8-3713b602fdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.531965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-be473213-2349-48f0-8cb8-3713b602fdfe None None] MFA Rules not processed for user `b00b6242e52a4ab1bc98280fe7242647`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:55.536538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be473213-2349-48f0-8cb8-3713b602fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:55.550593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:55.575604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 824/1647] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:55.597832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04dd7e-c990-48e3-ad5f-b304a13b4cbe None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.598442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04dd7e-c990-48e3-ad5f-b304a13b4cbe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.598863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04dd7e-c990-48e3-ad5f-b304a13b4cbe None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.600867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 825/1648] 10.222.0.73 () {54 vars in 961 bytes} [Thu May 12 17:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:44:55.614550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.614550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.614550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.648990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:55.651750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:55.708893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242f213-906d-4e47-ab56-194acca1fba7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:55.722895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:55.724410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.724682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.725134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] PATH_INFO: `/v3/users/8bd19c47ef9944c3b1791c61d513a5a2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.727334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd19c47ef9944c3b1791c61d513a5a2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:55.729269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: WARNING py.warnings [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:44:55.729269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: warnings.warn(msg) May 12 17:44:55.729269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: May 12 17:44:55.733475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:55.849341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 826/1649] 10.222.0.73 () {58 vars in 1058 bytes} [Thu May 12 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:55.850471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:55.865986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac5ea-25f3-43ac-a513-f8df220497dd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:55.877031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:55.877784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:55.878136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:55.878495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:55.899763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd19c47ef9944c3b1791c61d513a5a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.013869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd19c47ef9944c3b1791c61d513a5a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.015466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd19c47ef9944c3b1791c61d513a5a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.018601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd19c47ef9944c3b1791c61d513a5a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.024324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:56.026122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:56.041492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a9c17a15-9534-4815-84e8-21d3f972a29d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd19c47ef9944c3b1791c61d513a5a2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.050419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 827/1650] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:56.052267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:56.087219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 824/1651] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:44:55 2022] DELETE /identity/v3/users/8bd19c47ef9944c3b1791c61d513a5a2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:44:56.090498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d72cdb-681b-425d-98fa-f3c77c9e1bed service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:56.094752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.095357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.095859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.142714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:56.144603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:56.191499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:56.192945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.193600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.193972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.196572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:56.331934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 825/1652] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:56.339290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:56.340929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:56.360044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 828/1653] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:56.375872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:56.378170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:56.460472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:56.461975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:56.482869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d8a1e-56c2-4a6f-95a4-0820b93c97e5 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:56.495101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:56.495390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16537e6c-d9f1-4520-9667-9a6ae832f61b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:56.495982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.496297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.496560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] PATH_INFO: `/v3/users/cae2f818c44f4f70b38e0dda4b189bae` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.502155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae2f818c44f4f70b38e0dda4b189bae)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:56.504089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:56.505126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144b54f-e773-4df1-9014-36b8a658f185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:56.507936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b54f-e773-4df1-9014-36b8a658f185 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.509204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b54f-e773-4df1-9014-36b8a658f185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.510588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b54f-e773-4df1-9014-36b8a658f185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.541278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144b54f-e773-4df1-9014-36b8a658f185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:56.544104 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144b54f-e773-4df1-9014-36b8a658f185 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:56.575557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 826/1654] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:56.601123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae2f818c44f4f70b38e0dda4b189bae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.616312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae2f818c44f4f70b38e0dda4b189bae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.641713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae2f818c44f4f70b38e0dda4b189bae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.642806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae2f818c44f4f70b38e0dda4b189bae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.656209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f70c6f7b-68cd-477f-a79d-74519f1378e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae2f818c44f4f70b38e0dda4b189bae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:56.665966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 829/1655] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:44:56 2022] DELETE /identity/v3/users/cae2f818c44f4f70b38e0dda4b189bae => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:44:56.667954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fc983-b69b-46d0-b047-10924d128f6d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.668749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fc983-b69b-46d0-b047-10924d128f6d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.669124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fc983-b69b-46d0-b047-10924d128f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.698714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-306fc983-b69b-46d0-b047-10924d128f6d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:56.700950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-306fc983-b69b-46d0-b047-10924d128f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:56.844215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 827/1656] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:56.870891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c454c-7cb8-46f8-9ade-36cc0811b48e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.871428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c454c-7cb8-46f8-9ade-36cc0811b48e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.871897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c454c-7cb8-46f8-9ade-36cc0811b48e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.874442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 830/1657] 10.222.0.73 () {54 vars in 912 bytes} [Thu May 12 17:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:44:56.910994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:56.911570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:56.912003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:56.942461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:56.944306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:57.008445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c454c-7cb8-46f8-9ade-36cc0811b48e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:57.010409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f51c454c-7cb8-46f8-9ade-36cc0811b48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:57.099062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 828/1658] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:57.111690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:57.113288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:57.165504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c454c-7cb8-46f8-9ade-36cc0811b48e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:57.174785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de334db-ac81-488f-8686-785df0cb8daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:57.175444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de334db-ac81-488f-8686-785df0cb8daa service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:57.175790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de334db-ac81-488f-8686-785df0cb8daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:57.176074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de334db-ac81-488f-8686-785df0cb8daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:57.201253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de334db-ac81-488f-8686-785df0cb8daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:57.204854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b67bd-427e-4c7c-a9e8-d908701909e0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:57.208490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de334db-ac81-488f-8686-785df0cb8daa service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:57.216299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf5fa9-6907-4ba3-8dba-b06c3fa90154 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:57.218679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf5fa9-6907-4ba3-8dba-b06c3fa90154 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:57.219217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf5fa9-6907-4ba3-8dba-b06c3fa90154 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:57.219708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf5fa9-6907-4ba3-8dba-b06c3fa90154 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:57.237157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 831/1659] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:57.246152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf5fa9-6907-4ba3-8dba-b06c3fa90154 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:57.248196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf5fa9-6907-4ba3-8dba-b06c3fa90154 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:57.266657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 829/1660] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:44:58.177589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9bdad-eb2d-44cc-b48f-a2089194700e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.178079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9bdad-eb2d-44cc-b48f-a2089194700e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.178160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9bdad-eb2d-44cc-b48f-a2089194700e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.181516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 832/1661] 10.222.0.73 () {54 vars in 912 bytes} [Thu May 12 17:44:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:44:58.187957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.188618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.188618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.223860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9bdad-eb2d-44cc-b48f-a2089194700e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:58.233683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d2a9bdad-eb2d-44cc-b48f-a2089194700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.308861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9bdad-eb2d-44cc-b48f-a2089194700e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:58.323547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:58.325140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.325441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.326981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] PATH_INFO: `/v3/projects/627ccbef6d014765a87060bc4fb7bde3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.330532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=627ccbef6d014765a87060bc4fb7bde3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:58.333758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 12 17:44:58.333758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) May 12 17:44:58.333758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: May 12 17:44:58.338217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:58.381202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] MFA Rules not processed for user `6076a5f71eed4f8fa47d74a79d6fa709`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:58.383461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.404683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 830/1662] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:58.458002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627ccbef6d014765a87060bc4fb7bde3', 'request_id': 'req-778b7130-2e99-4c7a-b490-9bf08da55129'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.470873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:58.473764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.483864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627ccbef6d014765a87060bc4fb7bde3', 'request_id': 'req-778b7130-2e99-4c7a-b490-9bf08da55129'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.492737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627ccbef6d014765a87060bc4fb7bde3', 'request_id': 'req-778b7130-2e99-4c7a-b490-9bf08da55129'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.506356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599414f-2e32-41ed-92a9-d32524a57f97 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:58.521578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] The token cache is being invalidate because project 627ccbef6d014765a87060bc4fb7bde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:44:58.522001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.522983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.528691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 833/1663] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:44:58 2022] DELETE /identity/v3/projects/627ccbef6d014765a87060bc4fb7bde3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:44:58.560027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:58.579081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.651432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:58.652411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.652954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.653063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] PATH_INFO: `/v3/users/57026d850c044b578bcb991e03b76aee` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.660363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57026d850c044b578bcb991e03b76aee)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:58.662557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:58.756842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57026d850c044b578bcb991e03b76aee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.758248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b7130-2e99-4c7a-b490-9bf08da55129 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:58.781858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57026d850c044b578bcb991e03b76aee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.782685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57026d850c044b578bcb991e03b76aee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.783418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57026d850c044b578bcb991e03b76aee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.789040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:58.790411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.790905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.791398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] PATH_INFO: `/v3/users/585c74ff6d7246cf8b4bbe099c3ba986` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.792089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57026d850c044b578bcb991e03b76aee'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.794296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=585c74ff6d7246cf8b4bbe099c3ba986)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:58.800323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:58.812237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 831/1664] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:44:58 2022] DELETE /identity/v3/users/57026d850c044b578bcb991e03b76aee => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:58.813703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:58.815839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.887447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585c74ff6d7246cf8b4bbe099c3ba986'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.899441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585c74ff6d7246cf8b4bbe099c3ba986'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.912080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585c74ff6d7246cf8b4bbe099c3ba986'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.912926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585c74ff6d7246cf8b4bbe099c3ba986'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.923234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76e334-399e-401a-9481-1b5d1098a0a7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:58.929062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:58.929765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.930046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.930321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.932897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-866f320a-8976-4a2c-9eac-1605edcbbf93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585c74ff6d7246cf8b4bbe099c3ba986'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:58.933192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.945720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 834/1665] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:44:58 2022] DELETE /identity/v3/users/585c74ff6d7246cf8b4bbe099c3ba986 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:58.946315 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:58.946637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:58.947140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:58.974402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:44:58.975597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:58.976515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:58.977285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:58.989429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 832/1666] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:59.062974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 835/1667] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:44:59.098744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:59.101713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:59.116999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70345-6754-418c-8464-74d326ad56c4 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:59.285723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:59.285723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:59.285723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:59.286798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:59.333555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:59.335993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:59.352133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 833/1668] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:44:59.396114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:59.401138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:44:59.429576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28716e-f940-46fc-ab37-e6fe94255015 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:59.442878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:59.443919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:59.444355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:59.445067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] PATH_INFO: `/v3/projects/a3227d2362fb4a33ac4288176db85394` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:59.448708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3227d2362fb4a33ac4288176db85394)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:59.450688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:59.545811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3227d2362fb4a33ac4288176db85394', 'request_id': 'req-41792d52-df3e-4051-b24d-769239d55ca8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.557826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3227d2362fb4a33ac4288176db85394', 'request_id': 'req-41792d52-df3e-4051-b24d-769239d55ca8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.575736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3227d2362fb4a33ac4288176db85394', 'request_id': 'req-41792d52-df3e-4051-b24d-769239d55ca8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.601765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] The token cache is being invalidate because project a3227d2362fb4a33ac4288176db85394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:44:59.602223 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.605123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41792d52-df3e-4051-b24d-769239d55ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.606920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 836/1669] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:44:59 2022] DELETE /identity/v3/projects/a3227d2362fb4a33ac4288176db85394 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:44:59.612916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:44:59.626600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e86d8e-68aa-4854-a14a-3262017e61c7 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:44:59.804706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:44:59.805608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:44:59.805892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:44:59.806166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] PATH_INFO: `/v3/users/e5ff273123a04779bf536978b7c58623` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:44:59.808787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5ff273123a04779bf536978b7c58623)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:44:59.813462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:44:59.929785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5ff273123a04779bf536978b7c58623'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.964806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5ff273123a04779bf536978b7c58623'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.966592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5ff273123a04779bf536978b7c58623'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.968088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5ff273123a04779bf536978b7c58623'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:44:59.989019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-340142bb-1425-4f77-b140-67d093061a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5ff273123a04779bf536978b7c58623'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:00.040469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 834/1670] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:44:59 2022] DELETE /identity/v3/users/e5ff273123a04779bf536978b7c58623 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:00.523528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fe45-04cc-4d35-9750-d4835d78ddf9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:00.524033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fe45-04cc-4d35-9750-d4835d78ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:00.524100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fe45-04cc-4d35-9750-d4835d78ddf9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:00.525625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 837/1671] 10.222.0.73 () {54 vars in 960 bytes} [Thu May 12 17:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:45:00.533433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:00.533988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:00.536870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:00.560220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] MFA Rules not processed for user `3704ba4cfa7f4ad8b798636b01ebd95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:00.567306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:00.651312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 835/1672] 10.222.0.73 () {58 vars in 1057 bytes} [Thu May 12 17:45:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:00.667051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5fe45-04cc-4d35-9750-d4835d78ddf9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:00.684503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5fe45-04cc-4d35-9750-d4835d78ddf9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:00.693042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:00.694996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:00.696010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:00.696491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:00.698655 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:00.713931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:00.716314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca3be-b23b-416b-9c81-09d9f12c1ac7 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:00.767284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 838/1673] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:00.805933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:00.807792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:00.994589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606fd36-9f1d-483d-9f12-8d19eb50e165 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:01.003942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:01.004867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.005151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.005430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] PATH_INFO: `/v3/projects/8ca3768c1dcd44488f8859a4d4f1fd7c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.006815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca3768c1dcd44488f8859a4d4f1fd7c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:01.008962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:01.089177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca3768c1dcd44488f8859a4d4f1fd7c', 'request_id': 'req-7ecdddf8-8461-44b7-870c-ceb407bef3d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.101705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca3768c1dcd44488f8859a4d4f1fd7c', 'request_id': 'req-7ecdddf8-8461-44b7-870c-ceb407bef3d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.112534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca3768c1dcd44488f8859a4d4f1fd7c', 'request_id': 'req-7ecdddf8-8461-44b7-870c-ceb407bef3d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.140685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] The token cache is being invalidate because project 8ca3768c1dcd44488f8859a4d4f1fd7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:01.141327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.142446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7ecdddf8-8461-44b7-870c-ceb407bef3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.144495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 836/1674] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:00 2022] DELETE /identity/v3/projects/8ca3768c1dcd44488f8859a4d4f1fd7c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:01.149227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979e369-8f16-4106-b94e-475867712b12 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.149705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979e369-8f16-4106-b94e-475867712b12 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.150123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979e369-8f16-4106-b94e-475867712b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.180002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0979e369-8f16-4106-b94e-475867712b12 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:01.182410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0979e369-8f16-4106-b94e-475867712b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:01.276742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 839/1675] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:01.286263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.287848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.288359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.344520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:01.345760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:01.372030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 837/1676] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:01.376907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979e369-8f16-4106-b94e-475867712b12 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:01.387781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979e369-8f16-4106-b94e-475867712b12 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:01.402376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:01.403111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.403518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.403708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.405664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:01.406290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:01.416370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:01.446977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:01.463256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 840/1677] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:01.474993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de8188-0bd9-4ec5-92e2-1b21853b3c00 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:01.494387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:01.497474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.497474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.497474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.497474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:01.499302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:01.560150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:01.566507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:01.591194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 838/1678] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:01 2022] POST /identity/v3/projects => generated 391 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:01.597811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:01.619898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77de70-8777-4936-b008-a1c9d96639a1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:01.794868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f8a1a-dfba-402c-9d02-ab0885a0f6c7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:01.808740 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:01.809900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.810196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.810479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.811896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:01.813001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:01.813273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:01.813561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:01.813842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] PATH_INFO: `/v3/projects/2893bb227b564e909366a36266d9300e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:01.824194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2893bb227b564e909366a36266d9300e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:01.824395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:01.825510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:01.894201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 839/1679] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:01 2022] POST /identity/v3/users => generated 296 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:01.899273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:01.919148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1eebe-2200-47f8-88ed-f8b895da9b2a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:01.939594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2893bb227b564e909366a36266d9300e', 'request_id': 'req-6002ab07-01ee-4bc2-b539-76c58bfd1406'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.956064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2893bb227b564e909366a36266d9300e', 'request_id': 'req-6002ab07-01ee-4bc2-b539-76c58bfd1406'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:01.975388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2893bb227b564e909366a36266d9300e', 'request_id': 'req-6002ab07-01ee-4bc2-b539-76c58bfd1406'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.011655 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] The token cache is being invalidate because project 2893bb227b564e909366a36266d9300e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:02.012009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.012682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.021405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 841/1680] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:01 2022] DELETE /identity/v3/projects/2893bb227b564e909366a36266d9300e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:02.022936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:02.024506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.074911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b245be-969f-4837-9f94-ced93845f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:02.075867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b245be-969f-4837-9f94-ced93845f191 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.077260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b245be-969f-4837-9f94-ced93845f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.078689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b245be-969f-4837-9f94-ced93845f191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.080891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b245be-969f-4837-9f94-ced93845f191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:02.086344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b245be-969f-4837-9f94-ced93845f191 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:02.126941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 840/1681] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:02.130816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.131144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.131428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.153753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002ab07-01ee-4bc2-b539-76c58bfd1406 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:02.173418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:02.175200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.175866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:02.176806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.177057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.177489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] PATH_INFO: `/v3/projects/53cc45c6ac3e48a7aa2041ffd965d3db` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.179241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53cc45c6ac3e48a7aa2041ffd965d3db)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:02.181374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:02.205174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 841/1682] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:02.211237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:02.213421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.240640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff8147-833b-49c7-bc6e-48a072824f6a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:02.251364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84046b3e-3721-459a-b384-5a8494155be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:02.252063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84046b3e-3721-459a-b384-5a8494155be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.252497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84046b3e-3721-459a-b384-5a8494155be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.252943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84046b3e-3721-459a-b384-5a8494155be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.253872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84046b3e-3721-459a-b384-5a8494155be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:02.255486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84046b3e-3721-459a-b384-5a8494155be8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:02.286206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 842/1683] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:02.290851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b7f7b-1df5-4004-b7b6-f35bccb40380 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.291144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b7f7b-1df5-4004-b7b6-f35bccb40380 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.291459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b7f7b-1df5-4004-b7b6-f35bccb40380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.294003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53cc45c6ac3e48a7aa2041ffd965d3db', 'request_id': 'req-578468d3-f5c2-4a4e-be77-762dd9cb829b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.309147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53cc45c6ac3e48a7aa2041ffd965d3db', 'request_id': 'req-578468d3-f5c2-4a4e-be77-762dd9cb829b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.319135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53cc45c6ac3e48a7aa2041ffd965d3db', 'request_id': 'req-578468d3-f5c2-4a4e-be77-762dd9cb829b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.339515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ff7b7f7b-1df5-4004-b7b6-f35bccb40380 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:02.341059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ff7b7f7b-1df5-4004-b7b6-f35bccb40380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.351049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] The token cache is being invalidate because project 53cc45c6ac3e48a7aa2041ffd965d3db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:02.351346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.360687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:02.362469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 842/1684] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:02 2022] DELETE /identity/v3/projects/53cc45c6ac3e48a7aa2041ffd965d3db => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:02.363358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:02.364720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.509600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578468d3-f5c2-4a4e-be77-762dd9cb829b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:02.520221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 843/1685] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:02.524431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.524724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.525007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.535037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b90b-d5ed-4849-a384-5d95d68f251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:02.535959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b90b-d5ed-4849-a384-5d95d68f251b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.536266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b90b-d5ed-4849-a384-5d95d68f251b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.536545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b90b-d5ed-4849-a384-5d95d68f251b admin admin] PATH_INFO: `/v3/projects/b2e8ef319c5e4104be6561b5b5d98f6c/users/0377123594aa4678b58d529ebefb8dac/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.570232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:02.571477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.587903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869b90b-d5ed-4849-a384-5d95d68f251b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e8ef319c5e4104be6561b5b5d98f6c, user_id=0377123594aa4678b58d529ebefb8dac, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:02.589176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 844/1686] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:02.590060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:02.594460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869b90b-d5ed-4849-a384-5d95d68f251b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:02.613984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bb998-9327-456f-88cc-60e54b7ca3fd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:02.627152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 843/1687] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:02 2022] PUT /identity/v3/projects/b2e8ef319c5e4104be6561b5b5d98f6c/users/0377123594aa4678b58d529ebefb8dac/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:02.628906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.629422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.630227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.658058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:02.659738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.720092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:02.720810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.721111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.721366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.722106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:02.722412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:02.724015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54c0ac-986c-44bf-9fc3-b71f711111cf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:02.737793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 845/1688] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:02.739126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.739559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.739856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.772935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] MFA Rules not processed for user `0377123594aa4678b58d529ebefb8dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:02.774465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:02.793253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 844/1689] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:02.794364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:02.812585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff5bad-0621-4847-a06f-b67282c034fd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:02.828805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:02.829941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:02.830184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:02.830472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:02.831182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:02.835307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:02.864091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 846/1690] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:02.864733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:02.892520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13420f8-d0f2-4821-ab22-16e1a37f650e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:02.907529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 845/1691] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:02.908750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.004350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c44aa1-6255-40f1-950f-b578ed15f1eb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.022556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.023428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.023718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.023996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.024732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.026729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.041835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.043213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.043551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.043840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.047475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:03.047929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.052484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.086380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 847/1692] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:03.087543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.108057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba55320-7e9f-4510-a038-0d790672b162 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.122057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 846/1693] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:02 2022] POST /identity/v3/projects => generated 393 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:03.122057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.124258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.124957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.125242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.125518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.126217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.128141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.210458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 848/1694] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:03 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:03.210458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.219767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e0589-2cf1-4b79-bd91-bb0bf83e4602 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.227817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.228877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.230110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.230636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.231501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.234369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcdf6-4d75-47a7-9054-bb6f1f07f485 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.236247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.251142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.252218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.252988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.253763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.254943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.258040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.292205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 847/1695] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:03 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:03.293022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 849/1696] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:03 2022] POST /identity/v3/users => generated 303 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:03.299478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.300691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.313204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb0bbc-c306-4b08-b6c9-bd6c9ae61bba admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.397421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.398842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.399129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.399416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.400704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.402827 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.434492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731768-88b8-4f08-aeaa-6e228f5f4d59 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.446006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 850/1697] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:03 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:03.449833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.459852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.460563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.460844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.461115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.463952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.467726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.470299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676723-e7b4-4eaa-8ac3-acdb5e9a698a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.486019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.486795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.487144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.487676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.488599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.490596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.493920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 848/1698] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:03.494989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.514594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 851/1699] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:03.517386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.519068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b670d-2821-4054-8c22-963b582cb69f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.532515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.534230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.534583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.536063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.538668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.540924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076608e-eb73-4ebc-9276-85ddef4c3426 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.541242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.554554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.555357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.556353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.556353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.556966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.562646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.568713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 849/1700] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:03.569772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.592670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 852/1701] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:03.593962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e8257-eaa8-4982-a1f6-219898510680 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.595282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.610008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.611681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.612101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.612533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] PATH_INFO: `/v3/projects/cd83d99de6ef49f2ba74eb7ea6e80958/users/8cd7d89babb34f178edd3189f2d73968/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.620938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693ae6-a96f-46af-8717-d7a60c36e76a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.639228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.639992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.640446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.640790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.641609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.646576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.649029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd83d99de6ef49f2ba74eb7ea6e80958, user_id=8cd7d89babb34f178edd3189f2d73968, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.657546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.672863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 853/1702] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:03.673957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.688893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 850/1703] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:03 2022] PUT /identity/v3/projects/cd83d99de6ef49f2ba74eb7ea6e80958/users/8cd7d89babb34f178edd3189f2d73968/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:03.689465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.700653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a2558-0728-4c6e-b88a-10188bc23f90 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.838919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd3b98-f3e7-4baf-9744-4364451033e3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:03.844996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.845985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dddf9-41a2-416c-b498-d5a18489dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:03.846106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.846398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.846676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] PATH_INFO: `/v3/projects/985253bdf4a748988a1f54b4c3c1bb52/users/f51665514ff64eacb9da5ddd0745b839/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.847982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dddf9-41a2-416c-b498-d5a18489dc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.848393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dddf9-41a2-416c-b498-d5a18489dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.848671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dddf9-41a2-416c-b498-d5a18489dc82 admin admin] PATH_INFO: `/v3/projects/288456fa7c9c4a01a7f3c167dacb6d1a/users/2e83691dc79f4454b5b0d0f9d3f3c279/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.875498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dddf9-41a2-416c-b498-d5a18489dc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288456fa7c9c4a01a7f3c167dacb6d1a, user_id=2e83691dc79f4454b5b0d0f9d3f3c279, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.879106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dddf9-41a2-416c-b498-d5a18489dc82 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.883495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985253bdf4a748988a1f54b4c3c1bb52, user_id=f51665514ff64eacb9da5ddd0745b839, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:03.886591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:03.915434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 854/1705] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:03 2022] PUT /identity/v3/projects/985253bdf4a748988a1f54b4c3c1bb52/users/f51665514ff64eacb9da5ddd0745b839/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:03.924296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 851/1705] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:03 2022] PUT /identity/v3/projects/288456fa7c9c4a01a7f3c167dacb6d1a/users/2e83691dc79f4454b5b0d0f9d3f3c279/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:03.926784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:03.930339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:03.930607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:03.938267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:03.994317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] MFA Rules not processed for user `2e83691dc79f4454b5b0d0f9d3f3c279`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:03.995373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:04.084110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d53c3-a142-4cc1-82d7-b800a4874938 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:04.098256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32e168-0d6f-4ac6-915f-204c4a4b1244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:04.099283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32e168-0d6f-4ac6-915f-204c4a4b1244 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.104938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32e168-0d6f-4ac6-915f-204c4a4b1244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.105410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32e168-0d6f-4ac6-915f-204c4a4b1244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.109196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32e168-0d6f-4ac6-915f-204c4a4b1244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:04.115403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32e168-0d6f-4ac6-915f-204c4a4b1244 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:04.154101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 855/1706] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:04.156966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9902c-6fce-4e7e-9ce1-5614ef865a18 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.157258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9902c-6fce-4e7e-9ce1-5614ef865a18 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.160463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9902c-6fce-4e7e-9ce1-5614ef865a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.255278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-24e9902c-6fce-4e7e-9ce1-5614ef865a18 None None] MFA Rules not processed for user `f51665514ff64eacb9da5ddd0745b839`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:04.257517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-24e9902c-6fce-4e7e-9ce1-5614ef865a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:04.295556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 852/1707] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:04.300985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:04.338176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874a820-89b3-4866-804a-6a6a687f4a30 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:04.354392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:04.355475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.355763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.356268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] PATH_INFO: `/v3/projects/cd83d99de6ef49f2ba74eb7ea6e80958/users/8cd7d89babb34f178edd3189f2d73968/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.365156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd83d99de6ef49f2ba74eb7ea6e80958, user_id=8cd7d89babb34f178edd3189f2d73968, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:04.367903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:04.408280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 853/1708] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:04 2022] PUT /identity/v3/projects/cd83d99de6ef49f2ba74eb7ea6e80958/users/8cd7d89babb34f178edd3189f2d73968/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:04.417577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:04.458932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 856/1709] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:04.460140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.465096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.465976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.562489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] MFA Rules not processed for user `8cd7d89babb34f178edd3189f2d73968`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:04.568589 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:04.630730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ced6-6618-411d-9885-b49e0669e4a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:04.638273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:04.643002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.643465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.643883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.645433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:04.647688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:04.706971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 854/1710] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:04 2022] POST /identity/v3/projects => generated 381 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:04.708094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:04.804649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c5df4-3980-4731-a495-c41165d63280 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:04.808989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 857/1711] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:04.809826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:04.812344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:04.812478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.812918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.813026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.813706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:04.823505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:04.839408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6213d-db92-4e85-b8b6-52098b41a238 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:04.854181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:04.855309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:04.855418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:04.855604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:04.856529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:04.862585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:04.912484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 855/1712] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:04 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:04.913878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:04.920699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:04.939296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 858/1713] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:04 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:04.940316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:04.962506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1a8b-4854-4148-a98f-6c5d9f939cb3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.075563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda438-234f-4f78-8518-e4dc42618bd1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.101802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.103435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.103435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.103435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.183543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.184439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.194626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.194922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.195678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.197655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.261587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 859/1714] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:04 2022] POST /identity/v3/users => generated 300 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:05.261587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.286583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.287931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734b825-47a2-4461-8f6a-83df104251f9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.289141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.313921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.319157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.319470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.319747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.320644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.328404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.329009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 856/1715] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:05.336142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.363111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 860/1716] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:05.364109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.378481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f263b8-c054-4205-863d-8fe24657725a service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.391478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f20a5-c494-42df-8f18-bb61411d22c0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.399993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.402195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.402646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.403581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.406542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.407436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.408515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.408811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.409086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] PATH_INFO: `/v3/projects/8a33862020044d9f869e42aa7e4a2c7a/users/25991615b71944348b89d780eaa3789c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.417661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.438930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a33862020044d9f869e42aa7e4a2c7a, user_id=25991615b71944348b89d780eaa3789c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.441007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.465081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 861/1717] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:05 2022] PUT /identity/v3/projects/8a33862020044d9f869e42aa7e4a2c7a/users/25991615b71944348b89d780eaa3789c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:05.467374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.469627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 857/1718] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:05.478725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.540735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a6645-6d1e-4af0-b0cd-ece042017a63 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.613130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21ffac-0930-42d0-9814-5a18e6a7c39e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.625107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.626352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.626783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.627230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.628111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.630054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.651463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 862/1719] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:05.652861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.743247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.747703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.747703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.747703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] PATH_INFO: `/v3/projects/74d099201fd547dea91fd7f5e9df3729/users/1880881171fa4d91a56aa214b6074edf/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.772264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee52c79-a8a9-4f8d-bd44-b6ac748006c1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:05.785099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea86987-90c7-46dd-b311-dfb009732b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:05.785685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d099201fd547dea91fd7f5e9df3729, user_id=1880881171fa4d91a56aa214b6074edf, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.786034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea86987-90c7-46dd-b311-dfb009732b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.786325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea86987-90c7-46dd-b311-dfb009732b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.786604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea86987-90c7-46dd-b311-dfb009732b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.788912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.819009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 858/1720] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:05 2022] PUT /identity/v3/projects/74d099201fd547dea91fd7f5e9df3729/users/1880881171fa4d91a56aa214b6074edf/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:05.821965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:05.909621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea86987-90c7-46dd-b311-dfb009732b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:05.913171 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea86987-90c7-46dd-b311-dfb009732b83 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:05.934061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 863/1721] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:05.936329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469ddd2-da47-4f37-9e48-b45416107504 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:05.936843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469ddd2-da47-4f37-9e48-b45416107504 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:05.937127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469ddd2-da47-4f37-9e48-b45416107504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:05.988997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4469ddd2-da47-4f37-9e48-b45416107504 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:05.990246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4469ddd2-da47-4f37-9e48-b45416107504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:06.018807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e30a9-10b3-48f8-ac6e-920aeb6dee62 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:06.036942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:06.038748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.041712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.042197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] PATH_INFO: `/v3/projects/8a33862020044d9f869e42aa7e4a2c7a/users/25991615b71944348b89d780eaa3789c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.050805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a33862020044d9f869e42aa7e4a2c7a, user_id=25991615b71944348b89d780eaa3789c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:06.064712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:06.112101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 859/1722] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:05 2022] PUT /identity/v3/projects/8a33862020044d9f869e42aa7e4a2c7a/users/25991615b71944348b89d780eaa3789c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:06.115798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:06.147298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 864/1723] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:06.148689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.148975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.149278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.230787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] MFA Rules not processed for user `25991615b71944348b89d780eaa3789c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:06.232055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:06.339702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf7e61-8706-4ddb-ae17-e3a5cd044618 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:06.353119 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:06.353851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.354143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.354424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.355224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:06.360364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:06.376596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 860/1724] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:06.377598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:06.391562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b0f74-66ff-4170-aa32-2ad6a76bd349 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:06.400147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0dcac-9890-4a8a-a044-755e27308d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:06.401441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dcac-9890-4a8a-a044-755e27308d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.401786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dcac-9890-4a8a-a044-755e27308d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.402116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dcac-9890-4a8a-a044-755e27308d86 admin admin] PATH_INFO: `/v3/projects/74d099201fd547dea91fd7f5e9df3729/users/1880881171fa4d91a56aa214b6074edf/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.405640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0dcac-9890-4a8a-a044-755e27308d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d099201fd547dea91fd7f5e9df3729, user_id=1880881171fa4d91a56aa214b6074edf, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:06.416759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0dcac-9890-4a8a-a044-755e27308d86 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:06.436461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 861/1725] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:06 2022] PUT /identity/v3/projects/74d099201fd547dea91fd7f5e9df3729/users/1880881171fa4d91a56aa214b6074edf/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:06.439001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.439497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.440105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.488055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] MFA Rules not processed for user `1880881171fa4d91a56aa214b6074edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:06.488386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 865/1726] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:06.489862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:06.506282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:06.507904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:06.638570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 862/1727] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:06.640128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:06.641917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:06.681392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418bf68-a7c5-462d-a8dd-45d0bc93d48d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:06.797582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:06.798274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.798554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.798828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.848802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fb323-8cf6-41c3-815c-8b208bd54455 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:06.864071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa6a00-aed9-4461-b745-221dadeeabd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:06.864835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa6a00-aed9-4461-b745-221dadeeabd4 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:06.865124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa6a00-aed9-4461-b745-221dadeeabd4 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:06.865405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa6a00-aed9-4461-b745-221dadeeabd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:06.965105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:06.966564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:06.997588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 866/1728] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:06.999027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:07.000774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa6a00-aed9-4461-b745-221dadeeabd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:07.003458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa6a00-aed9-4461-b745-221dadeeabd4 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:07.017451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddef39-1b84-423f-85c9-2e332f640369 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:07.027641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:07.028190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:07.028687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:07.028967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:07.047409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 863/1729] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:45:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:45:07.059789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:07.061842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:07.080686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 867/1730] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:07.143003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:07.143246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:07.143520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:07.162200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:07.303456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] MFA Rules not processed for user `7a2b51125e9645dc9454042f49ff3659`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:07.308821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:07.459658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9d53-25dd-43fb-a5ee-7bc6847795b8 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:07.493205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:07.494106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:07.495779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:07.496118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] PATH_INFO: `/v3/users/f51665514ff64eacb9da5ddd0745b839` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:07.499113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51665514ff64eacb9da5ddd0745b839)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:07.501895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:07.699007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 864/1731] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:07.699007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:07.739615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376858d-e24d-48b2-8d54-42a1877848b2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:07.740644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51665514ff64eacb9da5ddd0745b839'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:07.745063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f176bd-314f-483f-922f-afbb7e0e52e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:07.747603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f176bd-314f-483f-922f-afbb7e0e52e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:07.747923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f176bd-314f-483f-922f-afbb7e0e52e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:07.748858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f176bd-314f-483f-922f-afbb7e0e52e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:07.819529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51665514ff64eacb9da5ddd0745b839'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:07.828295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51665514ff64eacb9da5ddd0745b839'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:07.832533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51665514ff64eacb9da5ddd0745b839'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:07.872031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51665514ff64eacb9da5ddd0745b839'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:07.918031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f176bd-314f-483f-922f-afbb7e0e52e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:07.922161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 868/1732] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:07 2022] DELETE /identity/v3/users/f51665514ff64eacb9da5ddd0745b839 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:07.927309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:07.928336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f176bd-314f-483f-922f-afbb7e0e52e3 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:07.931143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:07.975694 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 865/1733] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:07.977289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:07.977581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:07.977862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:08.047754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:08.049033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:08.390310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa40bd-04f8-4436-9c97-789195435a79 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:08.423175 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:08.426135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:08.426708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:08.438991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:08.439330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 866/1734] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:08.446009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:08.583189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:08.585055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:08.780472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:08.801014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb44505-5b55-44e6-b3a4-f5f3e7bb877b service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:08.805548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c8dd2-5578-4553-8019-c757400eb26e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:08.820245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfae624-9014-4243-b548-852bcfd3a6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:08.823467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae624-9014-4243-b548-852bcfd3a6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:08.823800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae624-9014-4243-b548-852bcfd3a6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:08.824202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae624-9014-4243-b548-852bcfd3a6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:08.835924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 869/1735] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 917 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:08.858974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfae624-9014-4243-b548-852bcfd3a6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:08.858974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfae624-9014-4243-b548-852bcfd3a6c0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:08.906417 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 867/1736] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:10.701325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e7a08-1e49-435e-921e-8d4ed6c28389 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:10.701916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e7a08-1e49-435e-921e-8d4ed6c28389 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:10.701916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e7a08-1e49-435e-921e-8d4ed6c28389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:10.796965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d17e7a08-1e49-435e-921e-8d4ed6c28389 None None] MFA Rules not processed for user `298a13924a864f61adf6e4085e569358`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:10.806307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d17e7a08-1e49-435e-921e-8d4ed6c28389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:10.846566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:10.847937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:10.848396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:10.917415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] MFA Rules not processed for user `6076a5f71eed4f8fa47d74a79d6fa709`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:10.924487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:10.983824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 868/1737] 10.222.0.74 () {58 vars in 1015 bytes} [Thu May 12 17:45:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:11.035824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:11.037673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:11.057724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b68956-cd9f-4c16-bfc6-65ee54226978 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:11.079633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:11.081375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:11.083577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:11.088685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:11.120425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:11.125560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:11.150751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 870/1738] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:11.177662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:11.177957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:11.178275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:11.196245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 869/1739] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:11.276352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] MFA Rules not processed for user `298a13924a864f61adf6e4085e569358`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:11.289413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:11.333392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 871/1740] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:11.351350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:11.377193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454538cc-c699-4543-b513-83f94c5a80ad service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:11.399073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:11.399795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:11.400074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:11.400395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:11.408932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:11.410515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:11.455521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:11.457534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66b9d-4013-457c-bfbd-76b0836655ae None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:11.458401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:11.471720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:11.475855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:11.476148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:11.476455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] PATH_INFO: `/v3/projects/985253bdf4a748988a1f54b4c3c1bb52` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:11.486500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=985253bdf4a748988a1f54b4c3c1bb52)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:11.489151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 870/1741] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:11.492684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:11.656653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985253bdf4a748988a1f54b4c3c1bb52', 'request_id': 'req-235fbfdc-a685-4824-89c5-b6d154f2fbbc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:11.692848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985253bdf4a748988a1f54b4c3c1bb52', 'request_id': 'req-235fbfdc-a685-4824-89c5-b6d154f2fbbc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:11.703801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985253bdf4a748988a1f54b4c3c1bb52', 'request_id': 'req-235fbfdc-a685-4824-89c5-b6d154f2fbbc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:11.741867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] The token cache is being invalidate because project 985253bdf4a748988a1f54b4c3c1bb52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:11.742211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:11.745063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-235fbfdc-a685-4824-89c5-b6d154f2fbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:11.752286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 872/1742] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:11 2022] DELETE /identity/v3/projects/985253bdf4a748988a1f54b4c3c1bb52 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:11.754128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:11.771165 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b8ae0-f6e5-4541-82da-c74ead4dfaf7 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:12.019894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:12.021094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:12.021739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:12.022228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] PATH_INFO: `/v3/users/1880881171fa4d91a56aa214b6074edf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:12.025577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1880881171fa4d91a56aa214b6074edf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:12.038345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:12.178544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1880881171fa4d91a56aa214b6074edf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:12.197383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1880881171fa4d91a56aa214b6074edf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:12.225496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1880881171fa4d91a56aa214b6074edf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:12.226334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1880881171fa4d91a56aa214b6074edf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:12.266680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1880881171fa4d91a56aa214b6074edf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:12.289875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 871/1743] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:11 2022] DELETE /identity/v3/users/1880881171fa4d91a56aa214b6074edf => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:12.533872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:12.534242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:12.534524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:12.654602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] MFA Rules not processed for user `1cc16878bc6a445f8a66dfcdc54a7e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:12.671328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:12.901673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:12.903152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:12.952699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 873/1744] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:13.089665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d95d5-fe3b-42b1-9030-c69c57a0e885 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:13.103827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:13.105309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:13.105309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:13.108777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] PATH_INFO: `/v3/users/0377123594aa4678b58d529ebefb8dac` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:13.115180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0377123594aa4678b58d529ebefb8dac)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:13.121274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:13.225966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:13.227974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:13.301630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0377123594aa4678b58d529ebefb8dac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:13.340155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0377123594aa4678b58d529ebefb8dac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:13.368664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0377123594aa4678b58d529ebefb8dac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:13.369921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0377123594aa4678b58d529ebefb8dac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:13.431557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01ff5e33-e2bc-47d2-af00-dd32f8bf91d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0377123594aa4678b58d529ebefb8dac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:13.476602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 872/1745] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:12 2022] DELETE /identity/v3/users/0377123594aa4678b58d529ebefb8dac => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:13.486538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:13.496300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:13.501499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:13.591465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:13.594881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:13.609089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339a450-f297-4754-b914-6e0e7c6ebd38 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:13.768236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 873/1746] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:13.819173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:13.821026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:13.852064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:13.853324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:13.853879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:13.854315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:13.859580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:14.017167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:14.025311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:14.106277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf3c4b-d1a2-4b3d-8e4b-e27e30aa03b2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:14.109491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 874/1747] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 884 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:14.115740 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:14.124956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:14.129636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c9daf-a56a-4ca3-8c84-9e2bada19c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:14.135088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c9daf-a56a-4ca3-8c84-9e2bada19c24 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:14.135866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c9daf-a56a-4ca3-8c84-9e2bada19c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:14.136987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c9daf-a56a-4ca3-8c84-9e2bada19c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:14.211226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c9daf-a56a-4ca3-8c84-9e2bada19c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:14.221562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c9daf-a56a-4ca3-8c84-9e2bada19c24 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:14.222061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f1830-5dca-4e5f-861c-043e29ceb0db service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:14.239834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:14.244624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:14.244911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:14.245565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] PATH_INFO: `/v3/projects/74d099201fd547dea91fd7f5e9df3729` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:14.254275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d099201fd547dea91fd7f5e9df3729)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:14.259392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:14.263418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 874/1748] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:14.395699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d099201fd547dea91fd7f5e9df3729', 'request_id': 'req-36cd277c-6aa8-4e3a-9595-e7ebc255540d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:14.412910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d099201fd547dea91fd7f5e9df3729', 'request_id': 'req-36cd277c-6aa8-4e3a-9595-e7ebc255540d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:14.421065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d099201fd547dea91fd7f5e9df3729', 'request_id': 'req-36cd277c-6aa8-4e3a-9595-e7ebc255540d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:14.447706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] The token cache is being invalidate because project 74d099201fd547dea91fd7f5e9df3729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:14.447706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:14.447706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-36cd277c-6aa8-4e3a-9595-e7ebc255540d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:14.448105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:14.448105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:14.448105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:14.450019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 875/1749] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:14 2022] DELETE /identity/v3/projects/74d099201fd547dea91fd7f5e9df3729 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:14.461577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29c362-63ac-4924-821b-cf10b0ef0782 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:14.462034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29c362-63ac-4924-821b-cf10b0ef0782 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:14.462473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29c362-63ac-4924-821b-cf10b0ef0782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:14.474866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:14.476091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:14.517364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5c29c362-63ac-4924-821b-cf10b0ef0782 None None] MFA Rules not processed for user `298a13924a864f61adf6e4085e569358`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:14.520550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5c29c362-63ac-4924-821b-cf10b0ef0782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:14.754086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 876/1750] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:14.926308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5aea3-9208-4738-9f7d-de00edc10336 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:14.927508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5aea3-9208-4738-9f7d-de00edc10336 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:14.928127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5aea3-9208-4738-9f7d-de00edc10336 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:14.931201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 877/1751] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:45:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 12 17:45:14.988567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 875/1752] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:14.995785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:14.996072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:14.996377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.050688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:15.057281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:15.096085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 878/1753] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:15.097414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:15.109067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a4059-b792-46c2-96a2-2df3c747ee18 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:15.129393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:15.132295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.133256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.134045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.136329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:15.140500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:15.144702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:15.176107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 876/1754] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:15.177695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:15.195477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41034f9-f337-4768-bf38-2410fc36ee75 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:15.209332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765cadb-c2fb-4a38-95ba-47aa954bdfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:15.212347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765cadb-c2fb-4a38-95ba-47aa954bdfbc admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.212650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765cadb-c2fb-4a38-95ba-47aa954bdfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.212919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765cadb-c2fb-4a38-95ba-47aa954bdfbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.213641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765cadb-c2fb-4a38-95ba-47aa954bdfbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:15.222937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765cadb-c2fb-4a38-95ba-47aa954bdfbc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:15.289714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 879/1755] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:15 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:15.294529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:15.304536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ffd4c-170b-4450-ae4d-b5d971b34d86 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.304975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ffd4c-170b-4450-ae4d-b5d971b34d86 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.305770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ffd4c-170b-4450-ae4d-b5d971b34d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.308026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac6a6d-a7dc-4a1d-8a58-964dc4ea8179 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:15.348306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e94ffd4c-170b-4450-ae4d-b5d971b34d86 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:15.349631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e94ffd4c-170b-4450-ae4d-b5d971b34d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:15.578499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:15.585414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.588609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.591217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.593670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:15.598548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:15.622876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 880/1756] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:15.632216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.632380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.632652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.665064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 877/1757] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:15 2022] POST /identity/v3/users => generated 291 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:15.669743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:15.696143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cc2f3-41bb-4ad9-af30-04114cf2e714 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:15.718418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] MFA Rules not processed for user `7a2b51125e9645dc9454042f49ff3659`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:15.723929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:15.727400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.729062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.729570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.729833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:15.732245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:15.738691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:15.769880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 878/1758] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:15.781347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:15.787734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:15.852873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715ba7d-63b6-4d1f-8be5-347aad09f66b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:15.873568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:15.876300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:15.876440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:15.880731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:15.917427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:15.921739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:15.943549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 881/1759] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:15.949061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:15.977375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 879/1760] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:15.993254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa5af5-ad41-450b-b218-bd5437a44228 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:16.003223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607ac90-b75a-418b-be0a-c647ee4d2f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:16.007787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607ac90-b75a-418b-be0a-c647ee4d2f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:16.008076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607ac90-b75a-418b-be0a-c647ee4d2f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:16.008453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607ac90-b75a-418b-be0a-c647ee4d2f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:16.009243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607ac90-b75a-418b-be0a-c647ee4d2f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:16.014113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607ac90-b75a-418b-be0a-c647ee4d2f23 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:16.051382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 882/1761] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:16.055336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:16.075748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4082db7-7b07-4c53-bb1c-49ab3291381c service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:16.097779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:16.102654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:16.103944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:16.113077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] PATH_INFO: `/v3/projects/7eaf9b397baf46f7bacfe50e00d68cbc/users/d6b16dd50b29441dabfa5779c1e922e5/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:16.189390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eaf9b397baf46f7bacfe50e00d68cbc, user_id=d6b16dd50b29441dabfa5779c1e922e5, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:16.192732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:16.232854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 880/1762] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:16 2022] PUT /identity/v3/projects/7eaf9b397baf46f7bacfe50e00d68cbc/users/d6b16dd50b29441dabfa5779c1e922e5/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:16.244210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:16.244695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:16.245447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:16.317170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] MFA Rules not processed for user `d6b16dd50b29441dabfa5779c1e922e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:16.320521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:16.494753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 883/1763] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:16.499558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:16.742594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209382e2-8958-481b-8fd4-bf849f20ce3b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:16.780814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:16.781932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:16.782220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:16.782495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:16.783209 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:16.785190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:16.891703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 881/1764] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:16 2022] POST /identity/v3/projects => generated 381 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:16.895678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:16.920367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64610839-544b-4049-97f6-dfa10b5e0c45 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:17.142009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:17.143863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:17.178107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:17.179492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:17.184215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:17.184690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:17.187207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:17.197894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:17.250270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 884/1765] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:16 2022] POST /identity/v3/users => generated 297 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:17.254092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:17.275880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd859-1b64-4c80-806d-92892afa6b07 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:17.290527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:17.291297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:17.291656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:17.292028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:17.292887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:17.299043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:17.331636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 885/1766] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:17.334884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:17.372756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976b930-e5fa-4154-9057-31473c779214 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:17.398278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:17.402101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:17.411256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:17.411552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] PATH_INFO: `/v3/projects/dba032430f034e8ca6057521b37e39db/users/07f5a7344e8740bb8361fbf46a78abe8/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:17.427659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03763e-4107-45f0-9626-fe8d7f2eb3f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:17.461431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:17.462900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:17.463187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:17.463464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] PATH_INFO: `/v3/projects/b2e8ef319c5e4104be6561b5b5d98f6c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:17.473265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e8ef319c5e4104be6561b5b5d98f6c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:17.475318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:17.492201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba032430f034e8ca6057521b37e39db, user_id=07f5a7344e8740bb8361fbf46a78abe8, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:17.495040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:17.570169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 886/1767] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:17 2022] PUT /identity/v3/projects/dba032430f034e8ca6057521b37e39db/users/07f5a7344e8740bb8361fbf46a78abe8/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:17.582818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:17.746129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e8ef319c5e4104be6561b5b5d98f6c', 'request_id': 'req-911e6996-c050-48a9-a416-205ef01e2f38'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:17.767055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e8ef319c5e4104be6561b5b5d98f6c', 'request_id': 'req-911e6996-c050-48a9-a416-205ef01e2f38'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:17.781188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e8ef319c5e4104be6561b5b5d98f6c', 'request_id': 'req-911e6996-c050-48a9-a416-205ef01e2f38'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:17.806493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] The token cache is being invalidate because project b2e8ef319c5e4104be6561b5b5d98f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:17.807021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:17.815321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:17.818509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 882/1768] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:17 2022] DELETE /identity/v3/projects/b2e8ef319c5e4104be6561b5b5d98f6c => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:17.822259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:17.823933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:17.889853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf1dd0-0945-4dca-b0b3-45dd39df85ab admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:18.007894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6996-c050-48a9-a416-205ef01e2f38 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:18.018731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cf7d6-8ca4-4699-91f6-ffa61d69c22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:18.019280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cf7d6-8ca4-4699-91f6-ffa61d69c22d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.019566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cf7d6-8ca4-4699-91f6-ffa61d69c22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.019848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cf7d6-8ca4-4699-91f6-ffa61d69c22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.020647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cf7d6-8ca4-4699-91f6-ffa61d69c22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:18.022422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cf7d6-8ca4-4699-91f6-ffa61d69c22d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:18.025217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:18.026370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.026818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.027231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] PATH_INFO: `/v3/users/8cd7d89babb34f178edd3189f2d73968` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.035518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd7d89babb34f178edd3189f2d73968)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:18.039323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:18.043597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 887/1769] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:18.046724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534722da-9a69-4b31-9f6d-5198b1c54bf7 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.047182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534722da-9a69-4b31-9f6d-5198b1c54bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.047508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534722da-9a69-4b31-9f6d-5198b1c54bf7 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.054418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 888/1770] 10.222.0.73 () {54 vars in 912 bytes} [Thu May 12 17:45:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:45:18.055366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534722da-9a69-4b31-9f6d-5198b1c54bf7 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:18.075881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-534722da-9a69-4b31-9f6d-5198b1c54bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:18.161126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534722da-9a69-4b31-9f6d-5198b1c54bf7 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:18.171601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd7d89babb34f178edd3189f2d73968'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:18.176062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f144d1-6616-42e7-8f5b-df44b004e3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:18.180350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd7d89babb34f178edd3189f2d73968'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:18.180747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f144d1-6616-42e7-8f5b-df44b004e3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.183250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f144d1-6616-42e7-8f5b-df44b004e3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.183552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f144d1-6616-42e7-8f5b-df44b004e3c2 admin admin] PATH_INFO: `/v3/projects/dba032430f034e8ca6057521b37e39db/users/07f5a7344e8740bb8361fbf46a78abe8/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.198251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f144d1-6616-42e7-8f5b-df44b004e3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba032430f034e8ca6057521b37e39db, user_id=07f5a7344e8740bb8361fbf46a78abe8, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:18.199027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd7d89babb34f178edd3189f2d73968'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:18.199945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd7d89babb34f178edd3189f2d73968'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:18.214987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f144d1-6616-42e7-8f5b-df44b004e3c2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:18.243039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-041962e9-651b-4f3c-843a-bad812e75c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd7d89babb34f178edd3189f2d73968'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:18.272531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 889/1772] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:18 2022] PUT /identity/v3/projects/dba032430f034e8ca6057521b37e39db/users/07f5a7344e8740bb8361fbf46a78abe8/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:18.273195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 883/1772] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:17 2022] DELETE /identity/v3/users/8cd7d89babb34f178edd3189f2d73968 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:18.273285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.273285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.273285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.277298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.277588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.278342 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.348958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] MFA Rules not processed for user `6076a5f71eed4f8fa47d74a79d6fa709`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:18.354813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:18.405150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] MFA Rules not processed for user `07f5a7344e8740bb8361fbf46a78abe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:18.411143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:18.575339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 890/1773] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:45:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:18.639860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 884/1774] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:18.650805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:18.653483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:18.729087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:18.731404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:18.761825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee8b87-2152-4c23-afe4-5e5510699118 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:18.783487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:18.785007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.785351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.785678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.811182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:18.818539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:18.852447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 885/1775] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:18.890082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b56d-d6bd-4e74-87e3-669a6e57c46e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:18.909792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:18.912621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:18.912915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:18.914729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:18.954906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:18.958406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:18.991201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 891/1776] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:20.286115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:20.287530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:20.360341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:20.362189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:20.484451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577179fc-5feb-42ee-9b0a-8b0ef2b55def service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:20.491374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:20.493154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:20.493972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:20.494639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] PATH_INFO: `/v3/users/d6b16dd50b29441dabfa5779c1e922e5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:20.496603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b16dd50b29441dabfa5779c1e922e5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:20.502494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:20.527568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b79e5-eae7-4066-9abe-f02dd404cfc5 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:20.533280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:20.535993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:20.536556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:20.537025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] PATH_INFO: `/v3/projects/cd83d99de6ef49f2ba74eb7ea6e80958` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:20.538599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd83d99de6ef49f2ba74eb7ea6e80958)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:20.540770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:20.574292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b16dd50b29441dabfa5779c1e922e5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.586446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b16dd50b29441dabfa5779c1e922e5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.596600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b16dd50b29441dabfa5779c1e922e5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.597780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b16dd50b29441dabfa5779c1e922e5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.631237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-005346a2-9750-46e1-97a8-2f5439c4bcd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b16dd50b29441dabfa5779c1e922e5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.638993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd83d99de6ef49f2ba74eb7ea6e80958', 'request_id': 'req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.650540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 886/1777] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:20 2022] DELETE /identity/v3/users/d6b16dd50b29441dabfa5779c1e922e5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:20.652919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1566a-f41a-4d23-956e-832cd7ba1183 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:20.653242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1566a-f41a-4d23-956e-832cd7ba1183 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:20.653469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1566a-f41a-4d23-956e-832cd7ba1183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:20.660765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd83d99de6ef49f2ba74eb7ea6e80958', 'request_id': 'req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.690035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd83d99de6ef49f2ba74eb7ea6e80958', 'request_id': 'req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.724211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0ca1566a-f41a-4d23-956e-832cd7ba1183 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:20.726197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0ca1566a-f41a-4d23-956e-832cd7ba1183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:20.727151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] The token cache is being invalidate because project cd83d99de6ef49f2ba74eb7ea6e80958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:20.729029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.730087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:20.741596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 892/1778] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:20 2022] DELETE /identity/v3/projects/cd83d99de6ef49f2ba74eb7ea6e80958 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:20.742778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:20.746195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:20.881038 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 887/1779] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:20.883163 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:20.885903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:20.887146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:20.965354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:20.970117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:20.993017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf8e8a-00f3-4c1d-8ca0-afd0fd0b194f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:21.014889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 888/1780] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:21.020572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:21.022320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:21.023883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:21.024913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:21.025362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] PATH_INFO: `/v3/users/2e83691dc79f4454b5b0d0f9d3f3c279` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:21.026514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:21.028464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e83691dc79f4454b5b0d0f9d3f3c279)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:21.035420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:21.225336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2388b-eb10-4b36-8c86-11527fb8788b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:21.227364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e83691dc79f4454b5b0d0f9d3f3c279'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:21.241504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d94ae-c778-41f7-a55e-aebd50d57a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:21.244515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d94ae-c778-41f7-a55e-aebd50d57a3a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:21.244867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d94ae-c778-41f7-a55e-aebd50d57a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:21.246552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d94ae-c778-41f7-a55e-aebd50d57a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:21.284093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d94ae-c778-41f7-a55e-aebd50d57a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:21.291048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e83691dc79f4454b5b0d0f9d3f3c279'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:21.291801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e83691dc79f4454b5b0d0f9d3f3c279'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:21.291999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d94ae-c778-41f7-a55e-aebd50d57a3a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:21.292489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e83691dc79f4454b5b0d0f9d3f3c279'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:21.327675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1dea2817-f991-4e83-9e52-d46382e9866a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e83691dc79f4454b5b0d0f9d3f3c279'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:21.341148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 889/1781] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:21.346873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:21.348914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:21.349302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:21.381051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 893/1782] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:20 2022] DELETE /identity/v3/users/2e83691dc79f4454b5b0d0f9d3f3c279 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:21.389445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:21.389772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:21.390101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:21.453514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:21.454808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:21.470950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:21.474710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:21.657825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 890/1783] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:21.661260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:21.671747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 894/1784] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:21.687885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf7d57-0761-4c98-8d68-de3e5f641756 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:21.697693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:21.699061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:21.699629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:21.703418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:21.704485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:21.705132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:21.711681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:21.722843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:21.724746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 891/1785] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:21.725223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:21.729482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:21.743149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f7ff8-0ca5-4e6d-8b6c-a778adb6325e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:21.750584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:21.751565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:21.751999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:21.752734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:21.753554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:21.755763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:21.832115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 892/1786] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:21 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:21.836391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:22.017151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ea33-1ecb-43ca-93d2-a514cb01e0ea None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:22.168563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b37b2-55be-44f2-81ac-83bbc81b6d5c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:22.209739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:22.219487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:22.222224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:22.225025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:22.231462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:22.246060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:22.409584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:22.412585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:22.413855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:22.415045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:22.469833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 893/1787] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:21 2022] POST /identity/v3/users => generated 286 bytes in 634 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:22.470830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:22.492561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:22.496521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:22.550794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 895/1788] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:22.551845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbc2ca-c35a-4583-b60e-9337df621aa8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:22.570345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:22.571067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:22.571348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:22.571808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:22.572722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:22.574351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:22.627996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 894/1789] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:22.631644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:22.650961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf590e-d609-4472-a199-0eff9ce46adb service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:22.680237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcc13b-e0ea-4e80-b76e-9aaf7c4cc008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:22.681275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc13b-e0ea-4e80-b76e-9aaf7c4cc008 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:22.681663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc13b-e0ea-4e80-b76e-9aaf7c4cc008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:22.688604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcc13b-e0ea-4e80-b76e-9aaf7c4cc008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:22.689645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcc13b-e0ea-4e80-b76e-9aaf7c4cc008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:22.692388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcc13b-e0ea-4e80-b76e-9aaf7c4cc008 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:22.719315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 896/1790] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:22.723528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:22.742713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4d248-01f4-4054-bc8c-c4ad6dad681a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:22.768955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:22.769865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:22.770145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:22.770415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] PATH_INFO: `/v3/projects/c174b0b908d740c197dc6f68f52c4ae9/users/f7ca14aeb59a4b828b3cd8fd2c939f4a/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:22.827227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c174b0b908d740c197dc6f68f52c4ae9, user_id=f7ca14aeb59a4b828b3cd8fd2c939f4a, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:22.850008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:22.887074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 895/1791] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:22 2022] PUT /identity/v3/projects/c174b0b908d740c197dc6f68f52c4ae9/users/f7ca14aeb59a4b828b3cd8fd2c939f4a/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:22.899759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:22.900473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:22.900967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:22.978952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] MFA Rules not processed for user `f7ca14aeb59a4b828b3cd8fd2c939f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:22.980657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:23.181455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 897/1792] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:23.195325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:23.197955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:23.308342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:23.309840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:23.425477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69ce15-e51c-4d98-8f59-60828bca0e14 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:23.456791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:23.457718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:23.458120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:23.458525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:23.490360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:23.492555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:23.563306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 896/1793] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:23.564598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:23.591320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887e02f-0671-4cd0-a6c7-e43c596c5a2f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:23.614650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:23.615537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:23.617385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01ad91-9ac4-430c-88db-25535e8d2f04 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:23.621662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:23.622367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] PATH_INFO: `/v3/projects/7eaf9b397baf46f7bacfe50e00d68cbc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:23.633994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eaf9b397baf46f7bacfe50e00d68cbc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:23.646080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:23.647432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:23.648994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:23.651322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:23.651812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] PATH_INFO: `/v3/users/f7ca14aeb59a4b828b3cd8fd2c939f4a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:23.656448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ca14aeb59a4b828b3cd8fd2c939f4a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:23.662811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:23.807572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eaf9b397baf46f7bacfe50e00d68cbc', 'request_id': 'req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.808124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ca14aeb59a4b828b3cd8fd2c939f4a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.837532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eaf9b397baf46f7bacfe50e00d68cbc', 'request_id': 'req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.856121 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ca14aeb59a4b828b3cd8fd2c939f4a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.866316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eaf9b397baf46f7bacfe50e00d68cbc', 'request_id': 'req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.889744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ca14aeb59a4b828b3cd8fd2c939f4a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.890507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ca14aeb59a4b828b3cd8fd2c939f4a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.903857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] The token cache is being invalidate because project 7eaf9b397baf46f7bacfe50e00d68cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:23.905040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.911588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:23.926224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 898/1794] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:23 2022] DELETE /identity/v3/projects/7eaf9b397baf46f7bacfe50e00d68cbc => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:23.928084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:23.942288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:23.985066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ca14aeb59a4b828b3cd8fd2c939f4a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.024060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 897/1795] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:23 2022] DELETE /identity/v3/users/f7ca14aeb59a4b828b3cd8fd2c939f4a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:24.032256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:24.043557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:24.219760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2c541-47c4-4767-aaeb-ee3e4b0e0e4a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:24.411642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:24.413161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:24.413444 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:24.413722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] PATH_INFO: `/v3/users/07f5a7344e8740bb8361fbf46a78abe8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:24.427766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f5a7344e8740bb8361fbf46a78abe8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:24.440268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:24.489894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cc250-6135-43d1-872a-a1a2ef9b34e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:24.515992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:24.521470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:24.525401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:24.527272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] PATH_INFO: `/v3/projects/288456fa7c9c4a01a7f3c167dacb6d1a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:24.530783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] RBAC: Authorizing `identity:delete_project(project_id=288456fa7c9c4a01a7f3c167dacb6d1a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:24.536995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:24.611957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f5a7344e8740bb8361fbf46a78abe8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.658152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f5a7344e8740bb8361fbf46a78abe8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.662755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f5a7344e8740bb8361fbf46a78abe8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.664095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f5a7344e8740bb8361fbf46a78abe8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.708221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1af1eac7-65b0-4bf0-8ec7-34d584b3564a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f5a7344e8740bb8361fbf46a78abe8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.718124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288456fa7c9c4a01a7f3c167dacb6d1a', 'request_id': 'req-4a3064b7-40b3-462e-969f-53b6d105970a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.737227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288456fa7c9c4a01a7f3c167dacb6d1a', 'request_id': 'req-4a3064b7-40b3-462e-969f-53b6d105970a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.739410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 899/1796] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:23 2022] DELETE /identity/v3/users/07f5a7344e8740bb8361fbf46a78abe8 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:24.743885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:24.744215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:24.744492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:24.759972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288456fa7c9c4a01a7f3c167dacb6d1a', 'request_id': 'req-4a3064b7-40b3-462e-969f-53b6d105970a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.781467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:24.782323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:24.790665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] The token cache is being invalidate because project 288456fa7c9c4a01a7f3c167dacb6d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:24.791111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.792516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:24.799516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 898/1797] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:24 2022] DELETE /identity/v3/projects/288456fa7c9c4a01a7f3c167dacb6d1a => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:24.805586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:24.817156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:24.912376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3064b7-40b3-462e-969f-53b6d105970a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:24.922145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 900/1798] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:24.924026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:24.925444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:24.925942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:24.926543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] PATH_INFO: `/v3/users/25991615b71944348b89d780eaa3789c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:24.928025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25991615b71944348b89d780eaa3789c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:24.931608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:24.963339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:24.964918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:25.013497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25991615b71944348b89d780eaa3789c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:25.020868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25991615b71944348b89d780eaa3789c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:25.041370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25991615b71944348b89d780eaa3789c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:25.042180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25991615b71944348b89d780eaa3789c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:25.059604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25991615b71944348b89d780eaa3789c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:25.063651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4a42-0d38-4726-858e-953bd79f74e3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:25.070057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:25.070819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:25.071306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:25.071715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:25.072743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 899/1799] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:24 2022] DELETE /identity/v3/users/25991615b71944348b89d780eaa3789c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:25.074399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:25.089840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:25.092176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930da183-f9f9-45e6-8cfe-1a09d339c833 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:25.111592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 901/1800] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:26.095600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:26.098426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:26.161404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bc279-c5ef-4615-a5e3-dd4db9c1fa3c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.161995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bc279-c5ef-4615-a5e3-dd4db9c1fa3c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.161995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bc279-c5ef-4615-a5e3-dd4db9c1fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.193860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3e3bc279-c5ef-4615-a5e3-dd4db9c1fa3c None None] MFA Rules not processed for user `1cc16878bc6a445f8a66dfcdc54a7e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:26.196009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3e3bc279-c5ef-4615-a5e3-dd4db9c1fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:26.228566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d205ed-1944-4aa0-8ff3-a1033deebaa4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:26.234685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:26.236572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.237400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.237821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] PATH_INFO: `/v3/projects/dba032430f034e8ca6057521b37e39db` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.239550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba032430f034e8ca6057521b37e39db)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:26.244088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:26.312522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 902/1801] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:26.321869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba032430f034e8ca6057521b37e39db', 'request_id': 'req-534d32c2-3551-4515-b547-3ad51a84682e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:26.332415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba032430f034e8ca6057521b37e39db', 'request_id': 'req-534d32c2-3551-4515-b547-3ad51a84682e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:26.341153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba032430f034e8ca6057521b37e39db', 'request_id': 'req-534d32c2-3551-4515-b547-3ad51a84682e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:26.363725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] The token cache is being invalidate because project dba032430f034e8ca6057521b37e39db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:26.364135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:26.364922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534d32c2-3551-4515-b547-3ad51a84682e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:26.366759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 900/1802] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:26 2022] DELETE /identity/v3/projects/dba032430f034e8ca6057521b37e39db => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:26.373522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.373820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.374422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.411599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:26.412856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:26.539849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.541077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.541514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.546554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 903/1803] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:26.569932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:26.571390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:26.594448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 901/1804] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:26.600896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:26.611692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21831f-2944-44ae-a9a9-4d1d20209fba None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:26.617617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:26.618590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.619019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.619447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.620588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:26.621137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:26.623742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:26.647448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 904/1805] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:26.649435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:26.661882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd844179-9741-47d9-9b10-d8bddea06a91 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:26.671153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:26.671976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.672382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.672705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.673464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:26.675905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:26.713476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:26.724125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:26.724797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 902/1806] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:26 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:26.726582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:26.743305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:26.822211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27df2-707c-433d-840e-2674a8f5796c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:26.855333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4a5c2-a05c-42f5-b462-79398449359c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:26.866713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:26.867590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:26.867868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:26.868141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] PATH_INFO: `/v3/projects/8a33862020044d9f869e42aa7e4a2c7a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:26.870889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a33862020044d9f869e42aa7e4a2c7a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:26.876064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.019664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.020591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.022784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a33862020044d9f869e42aa7e4a2c7a', 'request_id': 'req-b33de8ba-1aae-487b-ac5f-d2163f2c9821'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.023592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.023984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] PATH_INFO: `/v3/projects/c174b0b908d740c197dc6f68f52c4ae9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.029079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c174b0b908d740c197dc6f68f52c4ae9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.032002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a33862020044d9f869e42aa7e4a2c7a', 'request_id': 'req-b33de8ba-1aae-487b-ac5f-d2163f2c9821'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.040203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a33862020044d9f869e42aa7e4a2c7a', 'request_id': 'req-b33de8ba-1aae-487b-ac5f-d2163f2c9821'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.042777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.066431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] The token cache is being invalidate because project 8a33862020044d9f869e42aa7e4a2c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:27.067047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.067936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.075140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 903/1807] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:26 2022] DELETE /identity/v3/projects/8a33862020044d9f869e42aa7e4a2c7a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:27.077278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.079017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.124447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c174b0b908d740c197dc6f68f52c4ae9', 'request_id': 'req-1c66dd43-f47a-4748-92e4-78c3f677a818'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.134708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c174b0b908d740c197dc6f68f52c4ae9', 'request_id': 'req-1c66dd43-f47a-4748-92e4-78c3f677a818'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.142697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c174b0b908d740c197dc6f68f52c4ae9', 'request_id': 'req-1c66dd43-f47a-4748-92e4-78c3f677a818'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.162883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33de8ba-1aae-487b-ac5f-d2163f2c9821 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.163798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] The token cache is being invalidate because project c174b0b908d740c197dc6f68f52c4ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:27.164214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.164863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1c66dd43-f47a-4748-92e4-78c3f677a818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:27.167444 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 905/1808] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:26 2022] DELETE /identity/v3/projects/c174b0b908d740c197dc6f68f52c4ae9 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:27.168928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b68190-86d7-46ec-8ff6-bc96162155ae None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.169447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b68190-86d7-46ec-8ff6-bc96162155ae None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.170422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b68190-86d7-46ec-8ff6-bc96162155ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.208623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f1b68190-86d7-46ec-8ff6-bc96162155ae None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:27.212501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f1b68190-86d7-46ec-8ff6-bc96162155ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.296101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8aaa4-6a63-4f23-b3c5-16e41e883163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.296863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8aaa4-6a63-4f23-b3c5-16e41e883163 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.297152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8aaa4-6a63-4f23-b3c5-16e41e883163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.297451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8aaa4-6a63-4f23-b3c5-16e41e883163 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.299230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8aaa4-6a63-4f23-b3c5-16e41e883163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.302230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8aaa4-6a63-4f23-b3c5-16e41e883163 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.308063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 906/1809] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:27.309500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.309792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.310123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.354984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:27.356624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 904/1810] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/users => generated 287 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:27.356765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.359968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf5d7-4a86-4811-8f5e-e80845414e7f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.360395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf5d7-4a86-4811-8f5e-e80845414e7f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.360720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf5d7-4a86-4811-8f5e-e80845414e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.370233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 907/1811] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:27.371533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.380546 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.390088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-83caf5d7-4a86-4811-8f5e-e80845414e7f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:27.391601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83caf5d7-4a86-4811-8f5e-e80845414e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.401319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 905/1812] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:27.406858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.407211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.407553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.419394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e447a-e86c-4b31-a19b-924cfef9d1ff None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.442575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.443397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.443860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.444301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.445240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.447038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.465674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:27.471765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 908/1813] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:27.472862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.475360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.499623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4ff9f-a214-47f0-87fd-17f91ae7fa0d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.504376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 906/1814] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:27.505612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.523430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.523913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.524555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.524825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.526808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:27.527221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.528986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.536662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2cd35-54bf-4ba8-a7ae-249111e130b5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.554613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.555331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.555595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.556192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.558293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.559603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.580307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 907/1815] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:27.581291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.588656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 909/1816] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:27.589670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.599229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f85c3-58af-4f2c-bc8e-900185c2d667 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.604078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.604796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.605073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.605347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.606079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:27.606384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.608633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.620075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc64656-8f6b-45dc-a82e-8851c260a5d2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.626675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.628107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.628591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.629154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] PATH_INFO: `/v3/projects/887261f594284422bcd448d443d27013/users/9ad11a30da01449f82c1b94c7181860f/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.630514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 908/1817] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:27.631462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.649458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad46674-aee4-4fd9-92d7-37e81c892071 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.663895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=887261f594284422bcd448d443d27013, user_id=9ad11a30da01449f82c1b94c7181860f, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.664516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e7fd5-ec04-4e1a-97b6-0ee0106610e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.665191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e7fd5-ec04-4e1a-97b6-0ee0106610e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.665471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e7fd5-ec04-4e1a-97b6-0ee0106610e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.665742 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e7fd5-ec04-4e1a-97b6-0ee0106610e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.666440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e7fd5-ec04-4e1a-97b6-0ee0106610e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.668074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.672206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e7fd5-ec04-4e1a-97b6-0ee0106610e7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.693593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 910/1818] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:27 2022] PUT /identity/v3/projects/887261f594284422bcd448d443d27013/users/9ad11a30da01449f82c1b94c7181860f/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:27.697293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.726317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 909/1819] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:27.726317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.726317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.726317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.767080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:27.768542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.785452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed6b6f-ec03-467c-86bf-ea43ad8b2d2d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.893180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.893927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.894212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.894528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.895225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.897431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.898874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 910/1820] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:27.900302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.916931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0ee3-8c7c-406d-8a6f-ff124f4f6d3c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:27.924981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:27.925806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:27.926140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:27.926574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:27.927882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:27.929499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:27.949849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 911/1821] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:27.951615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.954150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:27.955607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 911/1822] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/users => generated 291 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:27.956710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:27.969690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8e8f-120a-4717-8b37-25e3208f81e7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.053485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56188904-2318-47f6-ab10-ac0e24ff4d6c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.064014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.080199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.081164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.083763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.087137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.089955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.090239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.090515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.091889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.097527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.122388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.132536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.161148 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 912/1823] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:28.171488 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.174420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 912/1824] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:27 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:28.178426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.197725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b90c30-6b2d-40b3-aaad-b196fa9a62bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.199809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2a227-aa7c-4859-8194-8573759d0a9e service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.219688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.220593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.220593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.220784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.221366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.222977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.227202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.227886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.228288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.228472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.229202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.236114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.260140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 913/1825] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:28.263659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.270406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 913/1826] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:28.274969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.283996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41eb15-bb33-4715-961d-5e88b283bf9a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.297986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.298898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.299186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.299461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.302652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.302842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920767d-3980-43af-9495-3136b7d509d0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.304557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.318567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.319933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.319933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.319933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.326154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.327891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.358237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 914/1827] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:28.365291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.369822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 914/1828] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:28.370955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.386032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9fb11-c152-4278-9458-967bb70ed969 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.393751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89b863-b5ae-4619-bb8f-65ef24a94e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.394997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b863-b5ae-4619-bb8f-65ef24a94e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.395487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b863-b5ae-4619-bb8f-65ef24a94e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.396537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b863-b5ae-4619-bb8f-65ef24a94e2f admin admin] PATH_INFO: `/v3/projects/244a6451fbc34de9925135d76dd1a255/users/9e703b19610b4109b34467f7ddf8e5be/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.408616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062143b-b4d3-45a0-8876-a25554e82f41 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.427108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939158c-35a0-47b6-b6bc-6b1f3a7832f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.428849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939158c-35a0-47b6-b6bc-6b1f3a7832f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.429273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939158c-35a0-47b6-b6bc-6b1f3a7832f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.429686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939158c-35a0-47b6-b6bc-6b1f3a7832f0 admin admin] PATH_INFO: `/v3/projects/50ff64f252dd4a60af52618b077483f3/users/566d2efb251149228e64d67c0fe4733a/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.430687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89b863-b5ae-4619-bb8f-65ef24a94e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=244a6451fbc34de9925135d76dd1a255, user_id=9e703b19610b4109b34467f7ddf8e5be, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.437401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89b863-b5ae-4619-bb8f-65ef24a94e2f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.472310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 915/1829] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:28 2022] PUT /identity/v3/projects/244a6451fbc34de9925135d76dd1a255/users/9e703b19610b4109b34467f7ddf8e5be/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:28.472533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939158c-35a0-47b6-b6bc-6b1f3a7832f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ff64f252dd4a60af52618b077483f3, user_id=566d2efb251149228e64d67c0fe4733a, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.472533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939158c-35a0-47b6-b6bc-6b1f3a7832f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.474742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.475076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.475963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.505145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 915/1830] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:28 2022] PUT /identity/v3/projects/50ff64f252dd4a60af52618b077483f3/users/566d2efb251149228e64d67c0fe4733a/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:28.510102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] MFA Rules not processed for user `9e703b19610b4109b34467f7ddf8e5be`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:28.511313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:28.516375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.516618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.516873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.556119 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] MFA Rules not processed for user `566d2efb251149228e64d67c0fe4733a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:28.557358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:28.643514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 916/1831] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:28.645092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.691207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 916/1832] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:28.699462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.701955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:28.749238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b09d3e-8d8a-4046-89c6-a67ab18fcdaa None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.754682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.755412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.755696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.755969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.756889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.758820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.799410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d0233-0015-49db-98e3-267007d0f40a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.799743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 917/1833] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:28 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:28.802489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:28.863387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.864286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.864730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.864999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.950426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56b8d-8ccb-4b98-b5c3-ecd7f10a09c6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:28.969700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:28.971385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:28.971850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:28.972352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:28.974017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.975799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:28.977927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:28.979154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:29.014060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 917/1834] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:29.024572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 918/1835] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:28 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:29.030937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:29.043772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587dea-a1ab-4a7f-9cec-479f63d1627b service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:29.055734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:29.056650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:29.057174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:29.057307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:29.058266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:29.063037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:29.084037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 918/1836] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:29.084276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:29.100059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd158a-76dc-4b98-b59e-674ef4b7880b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:29.111370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:29.113724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:29.114155 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:29.114751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] PATH_INFO: `/v3/projects/b468632b167d4c83b7fd15a98e3a9696/users/dc9b951abed644d0bfa62440d00b64ad/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:29.136093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b468632b167d4c83b7fd15a98e3a9696, user_id=dc9b951abed644d0bfa62440d00b64ad, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:29.139335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:29.167012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 919/1837] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:29 2022] PUT /identity/v3/projects/b468632b167d4c83b7fd15a98e3a9696/users/dc9b951abed644d0bfa62440d00b64ad/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:29.171650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:29.185382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7765a5a-f0fb-4e31-9ee7-0b407d50abdb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:29.267296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3b34d-ba5e-4afe-9d63-92393fdb322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:29.267975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b34d-ba5e-4afe-9d63-92393fdb322e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:29.268320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b34d-ba5e-4afe-9d63-92393fdb322e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:29.268581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b34d-ba5e-4afe-9d63-92393fdb322e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:29.269627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3b34d-ba5e-4afe-9d63-92393fdb322e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:29.271705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3b34d-ba5e-4afe-9d63-92393fdb322e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:29.284373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 919/1838] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:29.288796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:29.303790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a139c-59a7-4ff1-9bd0-c7be943ce58c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:29.312022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:29.313170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:29.313630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:29.314094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] PATH_INFO: `/v3/projects/b468632b167d4c83b7fd15a98e3a9696/users/dc9b951abed644d0bfa62440d00b64ad/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:29.316638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b468632b167d4c83b7fd15a98e3a9696, user_id=dc9b951abed644d0bfa62440d00b64ad, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:29.319609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:29.336485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 920/1839] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:29 2022] PUT /identity/v3/projects/b468632b167d4c83b7fd15a98e3a9696/users/dc9b951abed644d0bfa62440d00b64ad/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:29.355024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5306255-a077-4513-ab9c-851ca4804bf3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:29.355561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5306255-a077-4513-ab9c-851ca4804bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:29.356043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5306255-a077-4513-ab9c-851ca4804bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:29.385475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e5306255-a077-4513-ab9c-851ca4804bf3 None None] MFA Rules not processed for user `dc9b951abed644d0bfa62440d00b64ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:29.386681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e5306255-a077-4513-ab9c-851ca4804bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:29.532238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 920/1840] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:29.668550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:29.670459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:29.762638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265ba84-e441-441d-a315-5d9eba944697 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:29.771979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218217-642d-4be7-89bb-7e9fc9275139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:29.773220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218217-642d-4be7-89bb-7e9fc9275139 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:29.773623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218217-642d-4be7-89bb-7e9fc9275139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:29.774615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218217-642d-4be7-89bb-7e9fc9275139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:29.790147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218217-642d-4be7-89bb-7e9fc9275139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:29.791898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218217-642d-4be7-89bb-7e9fc9275139 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:29.817863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 921/1841] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:30.354752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749089b-60a7-4ecb-9cd3-25bc9c15c3f8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:30.355857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749089b-60a7-4ecb-9cd3-25bc9c15c3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:30.356343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749089b-60a7-4ecb-9cd3-25bc9c15c3f8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:30.360018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 921/1842] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:45:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:45:30.446086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:30.446515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:30.446646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:30.528560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749089b-60a7-4ecb-9cd3-25bc9c15c3f8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:30.532988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:30.538100 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:30.555002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749089b-60a7-4ecb-9cd3-25bc9c15c3f8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:30.565270 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc9b951abed644d0bfa62440d00b64ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b468632b167d4c83b7fd15a98e3a9696', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:30.566551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:30.567105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:30.567585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:30.568575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:30.569105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:30.571407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:30.588276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 922/1843] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:30.593374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:30.603723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932d6c7-0a50-4de3-8248-8d3fecea67fa tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:30.609837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc9b951abed644d0bfa62440d00b64ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b468632b167d4c83b7fd15a98e3a9696', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:30.610833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:30.611306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:30.612526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:30.613442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:30.629119 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:30.661475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 923/1844] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:30 2022] POST /identity/v3/projects => generated 355 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:30.665135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:30.711373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 922/1845] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:45:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:30.739526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a814db2-7806-4b9c-b637-e6f319696e78 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:30.744136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc9b951abed644d0bfa62440d00b64ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b468632b167d4c83b7fd15a98e3a9696', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:30.744932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:30.745220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:30.745497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:30.746224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:30.746527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:30.748651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:30.762020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 924/1846] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:45:30.810050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:30.821843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff85912-55fc-4119-8d64-2ae100ceb735 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:30.827181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc9b951abed644d0bfa62440d00b64ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b468632b167d4c83b7fd15a98e3a9696', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:30.828141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:30.828474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:30.828747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] PATH_INFO: `/v3/projects/2e58e5ad5ae54dc0be7c14f6243df5a9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:30.830807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2e58e5ad5ae54dc0be7c14f6243df5a9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:30.833003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:30.887661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e58e5ad5ae54dc0be7c14f6243df5a9', 'request_id': 'req-008b8f59-29e0-4dcf-b617-88a23cfcc604'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:30.897929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:30.899825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:30.907614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e58e5ad5ae54dc0be7c14f6243df5a9', 'request_id': 'req-008b8f59-29e0-4dcf-b617-88a23cfcc604'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:30.916652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e58e5ad5ae54dc0be7c14f6243df5a9', 'request_id': 'req-008b8f59-29e0-4dcf-b617-88a23cfcc604'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:30.932728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] The token cache is being invalidate because project 2e58e5ad5ae54dc0be7c14f6243df5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:30.933015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:30.933694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-008b8f59-29e0-4dcf-b617-88a23cfcc604 tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:30.935914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 923/1847] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:30 2022] DELETE /identity/v3/projects/2e58e5ad5ae54dc0be7c14f6243df5a9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:31.023320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b09ea8-3bc6-4157-9def-dd101cd3b8bc tempest-VolumeQuotaClassesTest-1640694287 tempest-VolumeQuotaClassesTest-1640694287-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:31.027934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.028479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.029396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.080596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] MFA Rules not processed for user `9ad11a30da01449f82c1b94c7181860f`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:31.082115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.175791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:31.176948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.177754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.178827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.182289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.208664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:31.210714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:31.238284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 925/1848] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:31.253787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:31.254567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 924/1849] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:31.255586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.277269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:31.278826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.337823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6874f9-370d-45e2-a016-4a0479d34f09 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:31.340093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1464e-1661-445d-ab0f-7cc10f191469 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:31.343709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:31.344625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.344747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c10b-e152-44e8-b581-7b252c3d3a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:31.344938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.345174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] PATH_INFO: `/v3/users/9e703b19610b4109b34467f7ddf8e5be` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.345362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c10b-e152-44e8-b581-7b252c3d3a6c service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.345640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c10b-e152-44e8-b581-7b252c3d3a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.345912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c10b-e152-44e8-b581-7b252c3d3a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.346490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e703b19610b4109b34467f7ddf8e5be)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:31.348454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:31.354461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79c10b-e152-44e8-b581-7b252c3d3a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:31.355877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79c10b-e152-44e8-b581-7b252c3d3a6c service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:31.367999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 925/1850] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:31.399706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e703b19610b4109b34467f7ddf8e5be'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:31.409537 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e703b19610b4109b34467f7ddf8e5be'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:31.418314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e703b19610b4109b34467f7ddf8e5be'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:31.419158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e703b19610b4109b34467f7ddf8e5be'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:31.432302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e703b19610b4109b34467f7ddf8e5be'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:31.439771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 926/1851] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:31 2022] DELETE /identity/v3/users/9e703b19610b4109b34467f7ddf8e5be => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:31.444378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.444733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.445144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.463106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:31.464340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.527176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 926/1852] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:31.537159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:31.538665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.598355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33e792-cf7d-42da-8989-3a12182aca48 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:31.602054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:31.602731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.603012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.603285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.626827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:31.629855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:31.640583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 927/1853] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:31.914619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:31.916306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:31.953387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1fb9-2c39-4bca-8754-aac82093d3e2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:31.959596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:31.960687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:31.960969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:31.961244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] PATH_INFO: `/v3/projects/244a6451fbc34de9925135d76dd1a255` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:31.962452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244a6451fbc34de9925135d76dd1a255)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:31.964485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.020593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244a6451fbc34de9925135d76dd1a255', 'request_id': 'req-4762a562-22cb-4f65-92ce-f07c366ef962'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.027748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244a6451fbc34de9925135d76dd1a255', 'request_id': 'req-4762a562-22cb-4f65-92ce-f07c366ef962'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.034101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244a6451fbc34de9925135d76dd1a255', 'request_id': 'req-4762a562-22cb-4f65-92ce-f07c366ef962'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.045950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] The token cache is being invalidate because project 244a6451fbc34de9925135d76dd1a255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:32.046300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.046958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.048704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 927/1854] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:31 2022] DELETE /identity/v3/projects/244a6451fbc34de9925135d76dd1a255 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:32.052647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:32.063692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae68ae-d25b-4eed-8d7d-2bda1af38db5 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:32.126911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:32.127695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.127976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.128283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] PATH_INFO: `/v3/users/dc9b951abed644d0bfa62440d00b64ad` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.129464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc9b951abed644d0bfa62440d00b64ad)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:32.131347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.179700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc9b951abed644d0bfa62440d00b64ad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.187456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc9b951abed644d0bfa62440d00b64ad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.195213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc9b951abed644d0bfa62440d00b64ad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.195915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc9b951abed644d0bfa62440d00b64ad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.207572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9d772493-ce69-4240-84cc-ccc89c0d9dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc9b951abed644d0bfa62440d00b64ad'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.213451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 928/1855] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:32 2022] DELETE /identity/v3/users/dc9b951abed644d0bfa62440d00b64ad => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:32.487123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:32.488637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:32.502290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762a562-22cb-4f65-92ce-f07c366ef962 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:32.555961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:32.556844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.557126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.557397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] PATH_INFO: `/v3/projects/b468632b167d4c83b7fd15a98e3a9696` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.558568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b468632b167d4c83b7fd15a98e3a9696)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:32.561057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.604373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b468632b167d4c83b7fd15a98e3a9696', 'request_id': 'req-a7a3c191-e0fc-4106-899e-807d574e689f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.611784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b468632b167d4c83b7fd15a98e3a9696', 'request_id': 'req-a7a3c191-e0fc-4106-899e-807d574e689f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.617377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b468632b167d4c83b7fd15a98e3a9696', 'request_id': 'req-a7a3c191-e0fc-4106-899e-807d574e689f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.629449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] The token cache is being invalidate because project b468632b167d4c83b7fd15a98e3a9696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:32.629732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.630361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a7a3c191-e0fc-4106-899e-807d574e689f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:32.632040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 928/1856] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:32 2022] DELETE /identity/v3/projects/b468632b167d4c83b7fd15a98e3a9696 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:32.636810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.637262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.637359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.654751 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:32.655940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:32.720037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 929/1857] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:32.727970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.728301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.728590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.751518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:32.752755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:32.760652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 929/1858] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:32.766038 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:32.775651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4bbae-5cfe-4114-86c4-c3486ae71e8d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:32.780292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:32.780934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.781247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.781514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.782258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:32.782559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:32.784182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.795744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 930/1859] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:32.799799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:32.808760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6288ba8-1e0b-4662-98fe-0ffa2114bb74 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:32.813989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:32.814641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.814924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.815226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.816461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:32.817842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.844282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 930/1860] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:32 2022] POST /identity/v3/projects => generated 401 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:32.849096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:32.859891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cbee5-68a3-4fe9-bde5-6404ee69de83 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:32.926025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:32.927070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.927368 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.927664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.928396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:32.930552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.960510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 931/1861] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:32 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:32.961906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:32.972902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fc06b-3878-43dd-aa15-c9ada458a5db admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:32.978064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:32.978658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:32.979524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:32.980432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:32.981429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:32.984119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:32.998594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 931/1862] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:33.003116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.029743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e623c-332e-4539-9645-f291a7b677f1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:33.051720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e9558-20e1-4354-8df6-53362578e10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:33.052822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e9558-20e1-4354-8df6-53362578e10c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.053277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e9558-20e1-4354-8df6-53362578e10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.053720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e9558-20e1-4354-8df6-53362578e10c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.054482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e9558-20e1-4354-8df6-53362578e10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:33.056867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e9558-20e1-4354-8df6-53362578e10c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:33.080620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 932/1863] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:33.084696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.116819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2458-c0d8-4d57-bea3-f08076def4b0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:33.133292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:33.134195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.134478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.134756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] PATH_INFO: `/v3/projects/c90dff1ef8b64def9bd43a538f9c2356/users/99be30ed677f4587a04bc4a429d4aa1d/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.199141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90dff1ef8b64def9bd43a538f9c2356, user_id=99be30ed677f4587a04bc4a429d4aa1d, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:33.210262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:33.230845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 932/1864] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:33 2022] PUT /identity/v3/projects/c90dff1ef8b64def9bd43a538f9c2356/users/99be30ed677f4587a04bc4a429d4aa1d/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:33.242055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.242476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.243644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.278208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] MFA Rules not processed for user `99be30ed677f4587a04bc4a429d4aa1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:33.281237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:33.514703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 933/1865] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:33.519343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.677451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb072-b29b-4001-9365-cf4d1120e078 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:33.697497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f4e3-f47a-40a9-ab3f-584319fec10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:33.697497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f4e3-f47a-40a9-ab3f-584319fec10c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.697497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f4e3-f47a-40a9-ab3f-584319fec10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.697497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f4e3-f47a-40a9-ab3f-584319fec10c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.697916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f4e3-f47a-40a9-ab3f-584319fec10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:33.697916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f4e3-f47a-40a9-ab3f-584319fec10c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:33.734799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 933/1866] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:33 2022] POST /identity/v3/projects => generated 401 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:33.736130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.756764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695671aa-558a-418e-b4bb-b743f8c7add8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:33.800536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.800841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.801127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.817613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:33.818283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.818563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.818832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.818914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] MFA Rules not processed for user `9ad11a30da01449f82c1b94c7181860f`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:33.819613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:33.820082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:33.821553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:33.844814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 934/1867] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:33 2022] POST /identity/v3/users => generated 307 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:33.847103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.856348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc2b1-231c-4fff-9596-c84185df98f1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:33.861210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:33.861977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.862262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.862534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.863256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:33.865371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:33.882086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 935/1868] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:33.882296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 934/1869] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:33.883602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.898874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f11cb-7be0-4cf1-b256-64222159371d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:33.903597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:33.906046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:33.906395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:33.906733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] PATH_INFO: `/v3/projects/40a624c7ebce4f3a9be899e5dc54ee9a/users/7871adc58ee5461585d8a835772ad1ab/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:33.922517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a624c7ebce4f3a9be899e5dc54ee9a, user_id=7871adc58ee5461585d8a835772ad1ab, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:33.925322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:33.940759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 936/1870] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:33 2022] PUT /identity/v3/projects/40a624c7ebce4f3a9be899e5dc54ee9a/users/7871adc58ee5461585d8a835772ad1ab/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:33.945077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:33.956149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b03c65-cf9e-4cac-8d3b-ab475aad7188 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:34.013405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6cfe-4478-463b-ba09-7713224b13e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:34.014476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6cfe-4478-463b-ba09-7713224b13e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:34.014773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6cfe-4478-463b-ba09-7713224b13e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:34.015022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6cfe-4478-463b-ba09-7713224b13e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:34.015938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6cfe-4478-463b-ba09-7713224b13e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:34.017585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6cfe-4478-463b-ba09-7713224b13e8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:34.027710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 935/1871] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:34.032035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:34.042427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed63689-243d-4b18-80b3-3a6c97383347 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:34.047134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:34.048010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:34.048352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:34.048628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] PATH_INFO: `/v3/projects/40a624c7ebce4f3a9be899e5dc54ee9a/users/7871adc58ee5461585d8a835772ad1ab/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:34.050707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a624c7ebce4f3a9be899e5dc54ee9a, user_id=7871adc58ee5461585d8a835772ad1ab, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:34.053392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:34.073399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 937/1872] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:34 2022] PUT /identity/v3/projects/40a624c7ebce4f3a9be899e5dc54ee9a/users/7871adc58ee5461585d8a835772ad1ab/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:34.085689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:34.086012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:34.086292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:34.113537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] MFA Rules not processed for user `7871adc58ee5461585d8a835772ad1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:34.115043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:34.182933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 936/1873] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:34 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:34.194214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:34.196084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:34.262913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162db7c6-9084-4db4-ac52-116c8dd97564 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:34.267520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b8706-851f-4164-92d4-bdbcf99c23de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:34.268099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b8706-851f-4164-92d4-bdbcf99c23de service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:34.268429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b8706-851f-4164-92d4-bdbcf99c23de service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:34.268685 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b8706-851f-4164-92d4-bdbcf99c23de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:34.277999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b8706-851f-4164-92d4-bdbcf99c23de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:34.279457 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b8706-851f-4164-92d4-bdbcf99c23de service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:34.289817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 938/1874] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:34 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:34.370019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:34.381448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a339719-8ae3-4104-8907-1e63bba2fec6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:34.385091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d262b-4e60-4cbe-ba66-18c3869c6e07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:34.385735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d262b-4e60-4cbe-ba66-18c3869c6e07 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:34.386009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d262b-4e60-4cbe-ba66-18c3869c6e07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:34.386281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d262b-4e60-4cbe-ba66-18c3869c6e07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:34.479730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d262b-4e60-4cbe-ba66-18c3869c6e07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:34.481194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d262b-4e60-4cbe-ba66-18c3869c6e07 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:34.493304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 937/1875] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:38.870743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654fadd-041c-4fa9-b18c-7601b824f0a1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:38.871261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654fadd-041c-4fa9-b18c-7601b824f0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:38.871261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654fadd-041c-4fa9-b18c-7601b824f0a1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:38.877876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 939/1876] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:45:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:45:38.902422 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:38.902912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:38.902912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:38.954108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:38.958206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:39.147341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 938/1877] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:45:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:39.174447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654fadd-041c-4fa9-b18c-7601b824f0a1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:39.177679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0654fadd-041c-4fa9-b18c-7601b824f0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:39.244637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654fadd-041c-4fa9-b18c-7601b824f0a1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:39.261955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4a9c8-dab8-45b5-b2f7-0923217b926d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:39.264287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4a9c8-dab8-45b5-b2f7-0923217b926d service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:39.265055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4a9c8-dab8-45b5-b2f7-0923217b926d service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:39.266727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4a9c8-dab8-45b5-b2f7-0923217b926d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:39.295810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4a9c8-dab8-45b5-b2f7-0923217b926d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:39.301345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4a9c8-dab8-45b5-b2f7-0923217b926d service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:39.321029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 940/1878] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:40.285596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:40.368383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71307841-7f39-424f-a962-6dc06d21fdd5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:40.373824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:40.374830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:40.375210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:40.375609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] PATH_INFO: `/v3/users/99be30ed677f4587a04bc4a429d4aa1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:40.377024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] RBAC: Authorizing `identity:delete_user(user_id=99be30ed677f4587a04bc4a429d4aa1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:40.379092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:40.472001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99be30ed677f4587a04bc4a429d4aa1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:40.502959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99be30ed677f4587a04bc4a429d4aa1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:40.503731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99be30ed677f4587a04bc4a429d4aa1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:40.504450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99be30ed677f4587a04bc4a429d4aa1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:40.516319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99be30ed677f4587a04bc4a429d4aa1d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:40.537111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 939/1879] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:40 2022] DELETE /identity/v3/users/99be30ed677f4587a04bc4a429d4aa1d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:40.541907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:40.542427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:40.542664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:40.588321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:40.589807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:40.712491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 941/1880] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:40.757692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:40.759424 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:40.911734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3686ff1-f71e-4ae9-97d7-9d0ef523077d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:40.938067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:40.940353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:40.940648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:40.941683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:41.000193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:41.005415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:41.031461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 940/1881] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:41.790539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:41.792867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:41.813753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad6101-998a-4b0f-8e75-b585ae9cc1a8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:41.828629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:41.833385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:41.833867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:41.834320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] PATH_INFO: `/v3/projects/c90dff1ef8b64def9bd43a538f9c2356` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:41.835891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90dff1ef8b64def9bd43a538f9c2356)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:41.837868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:41.891756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90dff1ef8b64def9bd43a538f9c2356', 'request_id': 'req-00e352ba-2508-4013-97aa-885721f1680b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:41.900187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90dff1ef8b64def9bd43a538f9c2356', 'request_id': 'req-00e352ba-2508-4013-97aa-885721f1680b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:41.906502 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90dff1ef8b64def9bd43a538f9c2356', 'request_id': 'req-00e352ba-2508-4013-97aa-885721f1680b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:41.918697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] The token cache is being invalidate because project c90dff1ef8b64def9bd43a538f9c2356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:41.919063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:41.919774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:41.921647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 942/1882] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:41 2022] DELETE /identity/v3/projects/c90dff1ef8b64def9bd43a538f9c2356 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:41.925999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:41.938414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ee93-2fb1-4878-807c-37fffb393ff4 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:42.050982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:42.052378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:42.052844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:42.053388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] PATH_INFO: `/v3/users/7871adc58ee5461585d8a835772ad1ab` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:42.055026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7871adc58ee5461585d8a835772ad1ab)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:42.058390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:42.200060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7871adc58ee5461585d8a835772ad1ab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:42.229897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7871adc58ee5461585d8a835772ad1ab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:42.230875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7871adc58ee5461585d8a835772ad1ab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:42.232731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7871adc58ee5461585d8a835772ad1ab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:42.258541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7531d126-acd2-4ed7-ac47-3c5228719776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7871adc58ee5461585d8a835772ad1ab'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:42.291013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 941/1883] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:41 2022] DELETE /identity/v3/users/7871adc58ee5461585d8a835772ad1ab => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:43.126025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.128048 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:43.167278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e352ba-2508-4013-97aa-885721f1680b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.240762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.241617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.241898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.242172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] PATH_INFO: `/v3/projects/40a624c7ebce4f3a9be899e5dc54ee9a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.243408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a624c7ebce4f3a9be899e5dc54ee9a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.245670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.295490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a624c7ebce4f3a9be899e5dc54ee9a', 'request_id': 'req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:43.301926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a624c7ebce4f3a9be899e5dc54ee9a', 'request_id': 'req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:43.307431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a624c7ebce4f3a9be899e5dc54ee9a', 'request_id': 'req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:43.320954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] The token cache is being invalidate because project 40a624c7ebce4f3a9be899e5dc54ee9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:43.321243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:43.321863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-95328a83-d5b0-4d38-aec0-71e98a6d0bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:43.323395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 943/1884] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:43 2022] DELETE /identity/v3/projects/40a624c7ebce4f3a9be899e5dc54ee9a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:43.328880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.329350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.329534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.348711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:43.349917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:43.413389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 942/1885] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:43.421784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6f51-dc21-4070-be78-bae131204691 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.422068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6f51-dc21-4070-be78-bae131204691 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.422344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6f51-dc21-4070-be78-bae131204691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.441251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-208f6f51-dc21-4070-be78-bae131204691 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:43.442611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-208f6f51-dc21-4070-be78-bae131204691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:43.451738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 944/1886] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:43.455921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.465331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a724db8-4d0e-4024-9bbe-4cf4c8658511 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.469854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.470495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.470773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.471044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.471761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:43.472060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.473664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.485045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 943/1887] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:45:43.489156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f6f51-dc21-4070-be78-bae131204691 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.498998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f6f51-dc21-4070-be78-bae131204691 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.503533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.504190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.504487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.504762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.505455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.507329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.537604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 945/1888] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:43.539173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.549444 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c44aeb-dcf0-4a21-9d33-5ad9a7eae584 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.607616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.608327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.608640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.608933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.609629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.611543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.632050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 944/1889] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:43 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:43.636021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.646680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba4d4f-35e2-44fb-9545-c4d2b657f5ea admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.651239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.651885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.652184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.652461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.653176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.654789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.664790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 946/1890] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:43.668609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.681721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae6275-6f29-4caa-b5b2-e2b600deecfa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.687920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d72a3-ecf2-4214-9468-f030cfed17a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.688630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d72a3-ecf2-4214-9468-f030cfed17a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.688968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d72a3-ecf2-4214-9468-f030cfed17a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.689247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d72a3-ecf2-4214-9468-f030cfed17a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.689975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d72a3-ecf2-4214-9468-f030cfed17a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.691565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d72a3-ecf2-4214-9468-f030cfed17a5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.708650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 945/1891] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:43.709596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:43.719440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a20598-ef70-47dd-9f7f-376fc0addd02 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:43.724057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:43.724966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.725246 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.725519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] PATH_INFO: `/v3/projects/1c3a43f20cf34379a64e4d01add8da1c/users/581d9a6e82234cb7bceab02758105e6c/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.741228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3a43f20cf34379a64e4d01add8da1c, user_id=581d9a6e82234cb7bceab02758105e6c, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:43.743883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:43.758275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 947/1892] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:43 2022] PUT /identity/v3/projects/1c3a43f20cf34379a64e4d01add8da1c/users/581d9a6e82234cb7bceab02758105e6c/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:43.769357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:43.769937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:43.770217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:43.801388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] MFA Rules not processed for user `581d9a6e82234cb7bceab02758105e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:43.802082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:43.901861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 946/1893] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:43.907250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.017235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3c32f-c440-46d2-a857-be12713cd272 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.024098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.024965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.025376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.025755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.026555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.029245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.067747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 948/1894] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:43 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:44.068081 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.078213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cf08f-1a08-43a5-8846-7e05644b253a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.140361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.141020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.141300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.141572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.142252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.144173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.167145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 947/1895] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:44 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:44.168362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.178611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e76d7-489a-4472-b3bf-05ceadd76a58 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.184805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.185756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.186203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.186556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.187367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.189074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.205563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 949/1896] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:44.209431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.223889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5398d-2b88-4983-b4e1-52cde33b7837 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.228724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.229636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.229919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.230193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] PATH_INFO: `/v3/projects/d2468fc0779c4c6297dd435fb749a860/users/65918974d0504329ae3f5a484e03ac59/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.245168 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2468fc0779c4c6297dd435fb749a860, user_id=65918974d0504329ae3f5a484e03ac59, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.248215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.263675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 948/1897] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:44 2022] PUT /identity/v3/projects/d2468fc0779c4c6297dd435fb749a860/users/65918974d0504329ae3f5a484e03ac59/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:44.268620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.278294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9357-746e-4de3-8e16-78c1cb2d37be admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.347956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff64ac-efb8-42b4-884e-21fa8e1d4711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.348576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff64ac-efb8-42b4-884e-21fa8e1d4711 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.348858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff64ac-efb8-42b4-884e-21fa8e1d4711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.349130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff64ac-efb8-42b4-884e-21fa8e1d4711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.349854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff64ac-efb8-42b4-884e-21fa8e1d4711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.351465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff64ac-efb8-42b4-884e-21fa8e1d4711 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.361465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 950/1898] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:44.365839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.378442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dbdc0-80db-4da9-9eba-b9678c6e98d7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.390643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.391539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.391825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.392106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] PATH_INFO: `/v3/projects/d2468fc0779c4c6297dd435fb749a860/users/65918974d0504329ae3f5a484e03ac59/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.394373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2468fc0779c4c6297dd435fb749a860, user_id=65918974d0504329ae3f5a484e03ac59, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.397089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.409840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 949/1899] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:44 2022] PUT /identity/v3/projects/d2468fc0779c4c6297dd435fb749a860/users/65918974d0504329ae3f5a484e03ac59/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:44.421426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.421713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.421989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.448092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] MFA Rules not processed for user `65918974d0504329ae3f5a484e03ac59`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:44.449386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:44.521615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 951/1900] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:44.533274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:44.534932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:44.593868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f460a-afc5-4073-90af-5a8afebff9ee admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:44.597427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:44.598113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:44.598346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:44.598616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:44.655370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:44.656836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:44.665583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 950/1901] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:51.088558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:51.101068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5699f-a36d-44b4-a29d-8e862b269b4a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:51.105242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc9d4b-b1ef-4fcc-936b-b78b25aa6d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:51.106340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9d4b-b1ef-4fcc-936b-b78b25aa6d09 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:51.106740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9d4b-b1ef-4fcc-936b-b78b25aa6d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:51.107328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9d4b-b1ef-4fcc-936b-b78b25aa6d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:51.120423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc9d4b-b1ef-4fcc-936b-b78b25aa6d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:51.122011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc9d4b-b1ef-4fcc-936b-b78b25aa6d09 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:51.133027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 952/1902] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:51.353263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:51.354677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:51.432089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6edbd-736d-4d13-a41e-0d5065d9d7c4 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:51.438518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:51.439583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:51.439962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:51.440418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] PATH_INFO: `/v3/users/d3c959294173455f86249100dbcf6d07` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:51.441896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c959294173455f86249100dbcf6d07)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:51.444557 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:51.499525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c959294173455f86249100dbcf6d07'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:51.513949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c959294173455f86249100dbcf6d07'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:51.514948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c959294173455f86249100dbcf6d07'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:51.515646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c959294173455f86249100dbcf6d07'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:51.523229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c959294173455f86249100dbcf6d07'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:51.540532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 951/1903] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:51 2022] DELETE /identity/v3/users/d3c959294173455f86249100dbcf6d07 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:51.543304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:51.543595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:51.543875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:51.575213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:51.577481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:51.672895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 953/1904] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:51.686843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:51.688677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:51.803646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2506-e1d0-4a14-b17b-9c45788a6997 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:51.809181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:51.810781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:51.811299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:51.811739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:51.823576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:51.825241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:51.836995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 952/1905] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:52.220671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:52.222708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:52.244624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d5236-0889-4d4d-8392-a8975a531b31 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:52.250811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:52.251887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:52.252199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:52.252511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] PATH_INFO: `/v3/projects/c40177cf7be6471da409b9698dd5db5a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:52.253673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40177cf7be6471da409b9698dd5db5a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:52.255758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:52.319462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40177cf7be6471da409b9698dd5db5a', 'request_id': 'req-23520fe3-ed9f-473c-8b1b-e556f956a877'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.327786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40177cf7be6471da409b9698dd5db5a', 'request_id': 'req-23520fe3-ed9f-473c-8b1b-e556f956a877'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.342831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40177cf7be6471da409b9698dd5db5a', 'request_id': 'req-23520fe3-ed9f-473c-8b1b-e556f956a877'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.360215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] The token cache is being invalidate because project c40177cf7be6471da409b9698dd5db5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:52.360659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.361563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.367015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 954/1906] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:52 2022] DELETE /identity/v3/projects/c40177cf7be6471da409b9698dd5db5a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:52.368636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:52.379377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c6e59-fca9-4f05-9022-8c2085b7bda0 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:52.461699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:52.463174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:52.463651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:52.464088 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] PATH_INFO: `/v3/users/ab2e527fe8174ffe8e2bc6e4c43fbb5a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:52.465834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2e527fe8174ffe8e2bc6e4c43fbb5a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:52.468277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:52.543520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2e527fe8174ffe8e2bc6e4c43fbb5a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.584413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2e527fe8174ffe8e2bc6e4c43fbb5a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.585177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2e527fe8174ffe8e2bc6e4c43fbb5a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.595489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2e527fe8174ffe8e2bc6e4c43fbb5a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.624205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-19094c4b-55ff-4c19-92ad-22fbaacc0941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2e527fe8174ffe8e2bc6e4c43fbb5a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:52.655947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 953/1907] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:52 2022] DELETE /identity/v3/users/ab2e527fe8174ffe8e2bc6e4c43fbb5a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:53.897079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:53.900261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:53.918347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23520fe3-ed9f-473c-8b1b-e556f956a877 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:53.995079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:53.996290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:53.996565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:53.996980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] PATH_INFO: `/v3/projects/ed8681c7f61c4dcdb751b4ded0fe8eec` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:53.999337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8681c7f61c4dcdb751b4ded0fe8eec)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.001585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.084925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8681c7f61c4dcdb751b4ded0fe8eec', 'request_id': 'req-401947eb-8c78-40a9-88b3-51ee19422270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.091577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8681c7f61c4dcdb751b4ded0fe8eec', 'request_id': 'req-401947eb-8c78-40a9-88b3-51ee19422270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.097269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8681c7f61c4dcdb751b4ded0fe8eec', 'request_id': 'req-401947eb-8c78-40a9-88b3-51ee19422270'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.114155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] The token cache is being invalidate because project ed8681c7f61c4dcdb751b4ded0fe8eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:54.114443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.116609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-401947eb-8c78-40a9-88b3-51ee19422270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.125323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 955/1908] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:53 2022] DELETE /identity/v3/projects/ed8681c7f61c4dcdb751b4ded0fe8eec => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:54.125637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.126451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.126940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.157861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:54.159584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:54.273265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 954/1909] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:54.282102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.282807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.284124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.304043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:54.305530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:54.315046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 956/1910] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:54.318966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.343136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d49fe8-94df-4d0c-ad68-8f4b6aa78c3e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.350188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.350870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.351152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.351441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.352144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:54.352490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.354075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.369016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.369163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 955/1911] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:45:54.370811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:54.374045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.383981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859ae43-b199-46f8-9ca7-b2343f7b4a67 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.386212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa43a6a-b037-437b-82d2-bbc140359860 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.390624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.391590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.391899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.392421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] PATH_INFO: `/v3/users/566d2efb251149228e64d67c0fe4733a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.392616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.393556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.393846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.394124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.394285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566d2efb251149228e64d67c0fe4733a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.394811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.396714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.396841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.435622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 956/1912] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:54 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:54.440523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.473653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566d2efb251149228e64d67c0fe4733a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.485402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566d2efb251149228e64d67c0fe4733a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.505044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566d2efb251149228e64d67c0fe4733a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.507001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566d2efb251149228e64d67c0fe4733a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.526270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfb4db-da96-4c22-9c1b-35033499acf0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.529387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6236f8e9-2b18-4c8e-b474-3a48e3bf1a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566d2efb251149228e64d67c0fe4733a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:54.533121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.534053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.534549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.534997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.535886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.536563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 957/1913] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:54 2022] DELETE /identity/v3/users/566d2efb251149228e64d67c0fe4733a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:54.538734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.540635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.541123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.541547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.567380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:54.568968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:54.574419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 957/1914] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:54 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:54.579210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.581798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:54.631334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263ada4-e28e-47f5-8af9-ed639a06646a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.699776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 958/1915] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:54.710821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.713347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:54.747640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.748951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.749247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.749548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.750333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.752052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.766986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 958/1916] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:54.769214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.782800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b7e6a-9fe5-4a6f-8867-9810d9646e2a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.789076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.789919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.790794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.791087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.791825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.798706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.817094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 959/1917] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:54.821729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:54.834962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dd2d6-393c-4ae3-96b9-6befcbb6483f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.844692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ad7e-7d86-4f9b-8ae8-a9850040439d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:54.846188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.847373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.847830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.848379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] PATH_INFO: `/v3/projects/e883bea70b69449c92bbfac45b2f03ae/users/a1ce58d56c184611974206a3cc27e700/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.851597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a856063-3973-4778-812b-0a3d7bf04bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:54.852286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a856063-3973-4778-812b-0a3d7bf04bce service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.852569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a856063-3973-4778-812b-0a3d7bf04bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.852843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a856063-3973-4778-812b-0a3d7bf04bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.865313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a856063-3973-4778-812b-0a3d7bf04bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.867067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a856063-3973-4778-812b-0a3d7bf04bce service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.867574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e883bea70b69449c92bbfac45b2f03ae, user_id=a1ce58d56c184611974206a3cc27e700, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:54.870781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:54.879019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 959/1918] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:54.897236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 960/1919] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:54 2022] PUT /identity/v3/projects/e883bea70b69449c92bbfac45b2f03ae/users/a1ce58d56c184611974206a3cc27e700/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:54.914491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:54.915021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:54.915021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:54.972194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] MFA Rules not processed for user `a1ce58d56c184611974206a3cc27e700`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:54.973699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.070074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 960/1920] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:55.079186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.176382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2b1fa-dbd2-4a14-ab2c-7f87f6b6cc9f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.181341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.182008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.182294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.182571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.183265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.186048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.223671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 961/1921] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:55.225249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.253221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.262182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaae35-6600-4a89-a6ae-aece0b5bd887 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.265417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.385623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.386338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.386796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.387086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.389465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.391623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31d92b-c294-4be3-9f5a-f0d45ad472cd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.391949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.407523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.408945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.409403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.409795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] PATH_INFO: `/v3/projects/50ff64f252dd4a60af52618b077483f3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.412096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50ff64f252dd4a60af52618b077483f3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.414401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.416820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 961/1922] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:55.421151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.431749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa671f46-8a6a-45ec-be80-863f31274da4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.436471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.437132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.437415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.437690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.438422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.440041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.451136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 962/1923] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:55.455366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.464841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0ddda-e212-4a18-9860-51825e88888c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.467840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50ff64f252dd4a60af52618b077483f3', 'request_id': 'req-c5382fb3-9281-40ce-81d5-61478f254836'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:55.475707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50ff64f252dd4a60af52618b077483f3', 'request_id': 'req-c5382fb3-9281-40ce-81d5-61478f254836'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:55.482700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50ff64f252dd4a60af52618b077483f3', 'request_id': 'req-c5382fb3-9281-40ce-81d5-61478f254836'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:55.494236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] The token cache is being invalidate because project 50ff64f252dd4a60af52618b077483f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:55.494524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:55.495180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c5382fb3-9281-40ce-81d5-61478f254836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:55.496798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 962/1924] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:55 2022] DELETE /identity/v3/projects/50ff64f252dd4a60af52618b077483f3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:55.502388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50017-2139-449e-8a4c-74ea919af225 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.502675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50017-2139-449e-8a4c-74ea919af225 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.502952 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50017-2139-449e-8a4c-74ea919af225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.522111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-d9a50017-2139-449e-8a4c-74ea919af225 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:55.523310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d9a50017-2139-449e-8a4c-74ea919af225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.534115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.534997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.535285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.535546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] PATH_INFO: `/v3/projects/030901be9604439fb8a84f88f31f3376/users/7570aaa250b84f8b84384c52f2fc527c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.551218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030901be9604439fb8a84f88f31f3376, user_id=7570aaa250b84f8b84384c52f2fc527c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.554210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.569339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 963/1925] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:55 2022] PUT /identity/v3/projects/030901be9604439fb8a84f88f31f3376/users/7570aaa250b84f8b84384c52f2fc527c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:55.573185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.575206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.594424 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 963/1926] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:55.602229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.602596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.602840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.630271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:55.631543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.657649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89dcf-65dc-4946-adfd-fbd87ea55f52 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.662484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.663155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.663439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.663709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.664613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.666208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.678485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 964/1927] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:55.682425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.695658 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cea78-e9b7-400b-a28b-96162fbba860 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.702286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eb3e3-0d3c-46eb-8c60-180e9b83e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.703226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eb3e3-0d3c-46eb-8c60-180e9b83e567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.703496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eb3e3-0d3c-46eb-8c60-180e9b83e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.703767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eb3e3-0d3c-46eb-8c60-180e9b83e567 admin admin] PATH_INFO: `/v3/projects/030901be9604439fb8a84f88f31f3376/users/7570aaa250b84f8b84384c52f2fc527c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.706062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eb3e3-0d3c-46eb-8c60-180e9b83e567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030901be9604439fb8a84f88f31f3376, user_id=7570aaa250b84f8b84384c52f2fc527c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.709673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eb3e3-0d3c-46eb-8c60-180e9b83e567 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.721057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 964/1928] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:55.725159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.729393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 965/1929] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:55 2022] PUT /identity/v3/projects/030901be9604439fb8a84f88f31f3376/users/7570aaa250b84f8b84384c52f2fc527c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:55.740976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.741268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.741586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.746628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaa686-2b5f-437e-9181-857f98249ee8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.770250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] MFA Rules not processed for user `7570aaa250b84f8b84384c52f2fc527c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:55.771784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.811353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.812023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.812341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.812620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.813355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:55.813661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.815258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.830471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 965/1930] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:45:55.831678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.843779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f378b9-bbe7-44c5-b201-f1b3306a7b88 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.849304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.850106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 966/1931] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:55.850230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.850322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.850838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.851592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.857594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.862266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.864562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:55.900212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 966/1932] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:55.902123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:55.939282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a8759-8ece-4aaf-b512-7fa9ec4f3ef7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.966752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d18337-861b-4743-aacc-930095671d01 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:55.971854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.972687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.972971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.973249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:55.974093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:55.976214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:55.995965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fee04d-9e15-4161-b0e9-9a7fe9158627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:55.997146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fee04d-9e15-4161-b0e9-9a7fe9158627 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:55.997146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fee04d-9e15-4161-b0e9-9a7fe9158627 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:55.997962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fee04d-9e15-4161-b0e9-9a7fe9158627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.002812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 967/1933] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:55 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:56.004071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.014724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42c5e8-7083-4272-a03f-3ad4839e3aba admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.020039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.020838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.021123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.021532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.022378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.023978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.037544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 968/1934] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:56.038527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.048252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1546ba-7a46-4fd6-a73c-debb1d3d8f2d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.052949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.053591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.053902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.054181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.054948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.056606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.061174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fee04d-9e15-4161-b0e9-9a7fe9158627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.062680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fee04d-9e15-4161-b0e9-9a7fe9158627 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.067802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 969/1935] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:56.072185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.073076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 967/1936] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:56.083225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26224a6a-457f-4d08-8e2b-672f51b57da6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.088389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.089270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.089570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.089820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] PATH_INFO: `/v3/projects/fb6b3c3df57349d9b01cf6c84012e60d/users/e9772d11a9f145f981b7664b83cb8d49/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.110005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6b3c3df57349d9b01cf6c84012e60d, user_id=e9772d11a9f145f981b7664b83cb8d49, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.113150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.127254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 970/1937] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:56 2022] PUT /identity/v3/projects/fb6b3c3df57349d9b01cf6c84012e60d/users/e9772d11a9f145f981b7664b83cb8d49/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:56.137932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.139103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.139619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.179768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] MFA Rules not processed for user `e9772d11a9f145f981b7664b83cb8d49`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:56.181072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:56.262124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 968/1938] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:56.267257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.333394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4e2f-4033-4113-b88b-1d698734c5f4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.338856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.339604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.339876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.340153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.340947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.344143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.371873 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 971/1939] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:56 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:56.376727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.388396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b206-b9ca-4945-8eaf-0c2f77ce8650 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.465095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.465852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.466245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.466594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.467522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.470275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.493631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 969/1940] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:56 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:56.496903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.509329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f93d4-65fc-4424-84b9-ab0eccbe1b57 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.521505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.522293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.522669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.523038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.524020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.525761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.549959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 972/1941] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:56.550323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.561732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277435b-ca73-4fb5-89fa-980d539a6784 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.567636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.568773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.569479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.569479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] PATH_INFO: `/v3/projects/f27d4c0a694246439d2ee73086b53005/users/10312d841edd414fad7ecbdb51bd1383/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.595028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27d4c0a694246439d2ee73086b53005, user_id=10312d841edd414fad7ecbdb51bd1383, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.597853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.630667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 970/1942] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:56 2022] PUT /identity/v3/projects/f27d4c0a694246439d2ee73086b53005/users/10312d841edd414fad7ecbdb51bd1383/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:56.634456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.645519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0f801-8900-4a71-854c-b52370975620 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.660339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.660339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:56.732335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.733316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.733597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.734487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.734764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.736364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.737914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547ae1-2f73-4f27-b117-5ccea9dfffe2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.742696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.743553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.743831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.744105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] PATH_INFO: `/v3/users/581d9a6e82234cb7bceab02758105e6c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.745354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581d9a6e82234cb7bceab02758105e6c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.747112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 973/1943] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:56.748858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.752331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.761948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464e0b-7c63-4b8f-b3f9-7c5c8dec25c4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:56.767595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdc98a-d0a3-491a-84ac-18d3acf17366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:56.768577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdc98a-d0a3-491a-84ac-18d3acf17366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.768937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdc98a-d0a3-491a-84ac-18d3acf17366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.769278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdc98a-d0a3-491a-84ac-18d3acf17366 admin admin] PATH_INFO: `/v3/projects/f27d4c0a694246439d2ee73086b53005/users/10312d841edd414fad7ecbdb51bd1383/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.771517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdc98a-d0a3-491a-84ac-18d3acf17366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27d4c0a694246439d2ee73086b53005, user_id=10312d841edd414fad7ecbdb51bd1383, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:56.774329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdc98a-d0a3-491a-84ac-18d3acf17366 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:56.795771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 974/1944] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:56 2022] PUT /identity/v3/projects/f27d4c0a694246439d2ee73086b53005/users/10312d841edd414fad7ecbdb51bd1383/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:56.797906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015fae0-229e-423c-b380-d1d133586458 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.798194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015fae0-229e-423c-b380-d1d133586458 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.798473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015fae0-229e-423c-b380-d1d133586458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.802349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581d9a6e82234cb7bceab02758105e6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:56.810797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581d9a6e82234cb7bceab02758105e6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:56.820847 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581d9a6e82234cb7bceab02758105e6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:56.821633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581d9a6e82234cb7bceab02758105e6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:56.827695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f015fae0-229e-423c-b380-d1d133586458 None None] MFA Rules not processed for user `10312d841edd414fad7ecbdb51bd1383`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:56.828937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f015fae0-229e-423c-b380-d1d133586458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:56.835649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-af00971d-dd75-412f-849f-4f8604f81a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581d9a6e82234cb7bceab02758105e6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:56.845101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 971/1945] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:56 2022] DELETE /identity/v3/users/581d9a6e82234cb7bceab02758105e6c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:56.846962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4539498-037e-4f59-9259-c982767ad938 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:56.847314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4539498-037e-4f59-9259-c982767ad938 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:56.847688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4539498-037e-4f59-9259-c982767ad938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:56.869635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c4539498-037e-4f59-9259-c982767ad938 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:56.871223 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4539498-037e-4f59-9259-c982767ad938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:56.921319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 975/1946] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:56.932739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015fae0-229e-423c-b380-d1d133586458 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.934479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f015fae0-229e-423c-b380-d1d133586458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:56.950953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 972/1947] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:56.961534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4539498-037e-4f59-9259-c982767ad938 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:56.963118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4539498-037e-4f59-9259-c982767ad938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:57.022554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015fae0-229e-423c-b380-d1d133586458 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:57.063920 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4539498-037e-4f59-9259-c982767ad938 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:57.069289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:57.070491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:57.071000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:57.071480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:57.086158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:57.087593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:57.101590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 973/1948] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:57.109965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:57.111147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:57.111762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:57.112190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:57.115109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:57.185818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:57.187488 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:57.199511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 976/1949] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:57.554859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:57.556508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:57.575052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cccaaf-a6b0-4a81-9222-8c3bcb2d048b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:57.587366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:57.588469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:57.588748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:57.589018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] PATH_INFO: `/v3/projects/1c3a43f20cf34379a64e4d01add8da1c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:57.590247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3a43f20cf34379a64e4d01add8da1c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:57.592252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:57.668873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3a43f20cf34379a64e4d01add8da1c', 'request_id': 'req-398c2724-a717-475b-8154-4eef2df59931'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.688795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3a43f20cf34379a64e4d01add8da1c', 'request_id': 'req-398c2724-a717-475b-8154-4eef2df59931'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.696370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3a43f20cf34379a64e4d01add8da1c', 'request_id': 'req-398c2724-a717-475b-8154-4eef2df59931'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.711624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] The token cache is being invalidate because project 1c3a43f20cf34379a64e4d01add8da1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:57.712055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.712731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.714384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 974/1950] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:57 2022] DELETE /identity/v3/projects/1c3a43f20cf34379a64e4d01add8da1c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:57.718990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:57.730306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bc909-d976-4169-9c32-e0808eec7f53 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:57.795138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:57.796043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:57.796346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:57.796621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] PATH_INFO: `/v3/users/65918974d0504329ae3f5a484e03ac59` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:57.798378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=65918974d0504329ae3f5a484e03ac59)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:57.803520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:57.858756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65918974d0504329ae3f5a484e03ac59'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.870869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65918974d0504329ae3f5a484e03ac59'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.875180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65918974d0504329ae3f5a484e03ac59'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.875180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65918974d0504329ae3f5a484e03ac59'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.876254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:57.877743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:57.880342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-073ceee0-fa72-45f0-a747-8811a45329bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65918974d0504329ae3f5a484e03ac59'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.891415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c2724-a717-475b-8154-4eef2df59931 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:57.896655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 977/1951] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:57 2022] DELETE /identity/v3/users/65918974d0504329ae3f5a484e03ac59 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:57.896972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:57.897845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:57.898157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:57.898451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] PATH_INFO: `/v3/users/a1ce58d56c184611974206a3cc27e700` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:57.899658 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ce58d56c184611974206a3cc27e700)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:57.901825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:57.953167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ce58d56c184611974206a3cc27e700'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.961437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ce58d56c184611974206a3cc27e700'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.969174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ce58d56c184611974206a3cc27e700'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.969938 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ce58d56c184611974206a3cc27e700'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.982904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ce58d56c184611974206a3cc27e700'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:57.988789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 975/1952] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:57 2022] DELETE /identity/v3/users/a1ce58d56c184611974206a3cc27e700 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:57.993264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:57.993552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:57.993829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.013631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:58.014826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.078022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 978/1953] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:58.088025 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.089523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.148686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21b56c-3fc4-4696-b658-f2d9500d2b21 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.152613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53c0f-ff31-4d7e-b2b7-67736ae499c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.153298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53c0f-ff31-4d7e-b2b7-67736ae499c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.153714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53c0f-ff31-4d7e-b2b7-67736ae499c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.154002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53c0f-ff31-4d7e-b2b7-67736ae499c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.164327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f53c0f-ff31-4d7e-b2b7-67736ae499c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.166168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f53c0f-ff31-4d7e-b2b7-67736ae499c5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.176640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 976/1954] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:45:58.290687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.292295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.306228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281dcc7-1c60-4288-aaf9-ff4bafafd587 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.311991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.313419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.313706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.313978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] PATH_INFO: `/v3/projects/d2468fc0779c4c6297dd435fb749a860` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.315217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2468fc0779c4c6297dd435fb749a860)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.317343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.375833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2468fc0779c4c6297dd435fb749a860', 'request_id': 'req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.385283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2468fc0779c4c6297dd435fb749a860', 'request_id': 'req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.391529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2468fc0779c4c6297dd435fb749a860', 'request_id': 'req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.402293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] The token cache is being invalidate because project d2468fc0779c4c6297dd435fb749a860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:58.402583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.403224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.404833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 979/1955] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:58 2022] DELETE /identity/v3/projects/d2468fc0779c4c6297dd435fb749a860 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:58.410645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9390-822f-4009-bc12-2f6762a36f05 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.410937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9390-822f-4009-bc12-2f6762a36f05 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.411216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9390-822f-4009-bc12-2f6762a36f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.429674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e30e9390-822f-4009-bc12-2f6762a36f05 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:58.430827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e30e9390-822f-4009-bc12-2f6762a36f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.474442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.476113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.492314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 977/1956] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:58.501802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.503714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.504004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.522905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:58.524087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.531936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 978/1957] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:58.535929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.546871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de75339-35e3-4ab4-8f3f-2dde2fa530b6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.548900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc67721-fa30-4f6b-9284-7895de3e3d52 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.554245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.555299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.555524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.555723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.555855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] PATH_INFO: `/v3/projects/e883bea70b69449c92bbfac45b2f03ae` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.556661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.557015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e883bea70b69449c92bbfac45b2f03ae)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.557127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.557458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.558318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:58.558712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.560645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.561178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.573354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 979/1958] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:45:58.577939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.603243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63442bb-69a4-4587-ac4c-6d075042efd4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.628140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e883bea70b69449c92bbfac45b2f03ae', 'request_id': 'req-06a5e737-aa3d-4cf5-afcb-6620925a9d28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.643036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e883bea70b69449c92bbfac45b2f03ae', 'request_id': 'req-06a5e737-aa3d-4cf5-afcb-6620925a9d28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.649769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e883bea70b69449c92bbfac45b2f03ae', 'request_id': 'req-06a5e737-aa3d-4cf5-afcb-6620925a9d28'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.663161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] The token cache is being invalidate because project e883bea70b69449c92bbfac45b2f03ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:58.663537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.664383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.666145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 980/1959] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:58 2022] DELETE /identity/v3/projects/e883bea70b69449c92bbfac45b2f03ae => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:58.671425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.672494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.675487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.676150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.676453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.676727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.677589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.680154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.685714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e737-aa3d-4cf5-afcb-6620925a9d28 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.691128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.692006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.692333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.692614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] PATH_INFO: `/v3/users/7570aaa250b84f8b84384c52f2fc527c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.693837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7570aaa250b84f8b84384c52f2fc527c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.695772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.712680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 980/1960] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:58 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:58.717384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.719801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.758563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7570aaa250b84f8b84384c52f2fc527c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.773878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7570aaa250b84f8b84384c52f2fc527c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.774622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7570aaa250b84f8b84384c52f2fc527c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.775485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7570aaa250b84f8b84384c52f2fc527c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.782947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7570aaa250b84f8b84384c52f2fc527c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:58.803135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 981/1961] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:58 2022] DELETE /identity/v3/users/7570aaa250b84f8b84384c52f2fc527c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:58.814704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518a2f4-f26a-49ce-a644-fdc956e7d768 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.895257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.895711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.895989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.896460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.897019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.898942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.936852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 981/1962] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:58 2022] POST /identity/v3/users => generated 281 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:58.938086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:58.939588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:58.954502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75c276-c616-4025-8651-5b09bd66c405 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:58.960890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:58.961631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:58.961970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:58.962328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:58.963253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:58.965262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:58.989463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 982/1963] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:58.990556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.001925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82702722-a404-4b5d-87dd-a5f3ed45d744 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.007007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f3e92-4167-4968-8850-b9e5f77086dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.007618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3e92-4167-4968-8850-b9e5f77086dc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.007900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3e92-4167-4968-8850-b9e5f77086dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.008207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3e92-4167-4968-8850-b9e5f77086dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.008965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3e92-4167-4968-8850-b9e5f77086dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.010557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3e92-4167-4968-8850-b9e5f77086dc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.022450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 982/1964] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:59.026626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.037658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d3bf2-cb17-40f2-8912-12a5be971187 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.047482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.048688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.049044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.049758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] PATH_INFO: `/v3/projects/68a2df7310e74b4496876dfe2812d5e1/users/41bc1d7df42048c381f0da477daea4ec/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.069785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a2df7310e74b4496876dfe2812d5e1, user_id=41bc1d7df42048c381f0da477daea4ec, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.072683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.088860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 983/1965] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:59 2022] PUT /identity/v3/projects/68a2df7310e74b4496876dfe2812d5e1/users/41bc1d7df42048c381f0da477daea4ec/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:59.101906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.102343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.102816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.134335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] MFA Rules not processed for user `41bc1d7df42048c381f0da477daea4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:59.135555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.166202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.167932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.245909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 983/1966] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:59.248419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c8332-233e-4378-9947-d8186787ddb2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.256723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.264224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.264224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.264224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] PATH_INFO: `/v3/projects/030901be9604439fb8a84f88f31f3376` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.264224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030901be9604439fb8a84f88f31f3376)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.264224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.288480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.291093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.332956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030901be9604439fb8a84f88f31f3376', 'request_id': 'req-4857e522-6227-460e-aaae-73055d6774f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.343984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030901be9604439fb8a84f88f31f3376', 'request_id': 'req-4857e522-6227-460e-aaae-73055d6774f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.350901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030901be9604439fb8a84f88f31f3376', 'request_id': 'req-4857e522-6227-460e-aaae-73055d6774f3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.358363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61437698-15b9-4a82-858a-5a9992eed05c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.365009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] The token cache is being invalidate because project 030901be9604439fb8a84f88f31f3376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:45:59.365377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.366205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4857e522-6227-460e-aaae-73055d6774f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.368112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 984/1967] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:45:59 2022] DELETE /identity/v3/projects/030901be9604439fb8a84f88f31f3376 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:45:59.373580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17444da-b7e1-4ba9-af8b-3dc7b1c043f6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.374629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17444da-b7e1-4ba9-af8b-3dc7b1c043f6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.375056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17444da-b7e1-4ba9-af8b-3dc7b1c043f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.398962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b17444da-b7e1-4ba9-af8b-3dc7b1c043f6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:59.400203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b17444da-b7e1-4ba9-af8b-3dc7b1c043f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.435484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.436503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.436936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.437474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.440956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.485782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 985/1968] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:59.497101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.499006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.499302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.513386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.514946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.518337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:59.519752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.525096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 984/1969] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:59.534172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 986/1970] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:45:59.535230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.540204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.541823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.548796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24756519-901a-4cd2-8371-07e63a3617a2 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.554268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.554982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.555265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.555538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.556326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:45:59.556445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f763fc-3156-433b-9d6e-38c5ac70b64a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.556614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.558395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.563023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.563913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.564252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.564634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] PATH_INFO: `/v3/users/41bc1d7df42048c381f0da477daea4ec` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.566312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bc1d7df42048c381f0da477daea4ec)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.568350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.572018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 985/1971] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:45:59.577974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.593349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f259a2c-fe65-4b45-9b9d-6f7c6ec89045 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.598686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.599390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.599673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.599949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.600681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.602663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.640099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bc1d7df42048c381f0da477daea4ec'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.659014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 986/1972] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:45:59.660336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.668769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bc1d7df42048c381f0da477daea4ec'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.670838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bc1d7df42048c381f0da477daea4ec'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.671485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bc1d7df42048c381f0da477daea4ec'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.681399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e5b2f85-cf11-42b3-9242-7bf841cc20d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bc1d7df42048c381f0da477daea4ec'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:45:59.696147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 987/1973] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:45:59 2022] DELETE /identity/v3/users/41bc1d7df42048c381f0da477daea4ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:45:59.700695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.700987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.701274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.720409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:45:59.721447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.734896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f8e1-7a6a-4d5d-821e-b91503380ed0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.789994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 988/1974] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:45:59.799833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.800112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.800843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.801097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.801375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.801804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.802069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.804150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.826856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 987/1975] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:45:59.830702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.832459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:45:59.845416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a581c7-7a3b-4f16-a5ee-ded502dd65d2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.850300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.851111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.851456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.851992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.852805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.854756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.869641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 988/1976] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:45:59.870780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.871624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e886de-5d0b-4a9e-965c-2a3713d4dac1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.875710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.876430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.876714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.877416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.884711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49baddf-2a09-48cf-bda1-3d23ddbd048b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.886591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.889474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.893898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91658f09-ac7f-434f-9fa0-e1199b35ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.894892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658f09-ac7f-434f-9fa0-e1199b35ac79 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.895327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658f09-ac7f-434f-9fa0-e1199b35ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.896400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658f09-ac7f-434f-9fa0-e1199b35ac79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.897313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91658f09-ac7f-434f-9fa0-e1199b35ac79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.900084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91658f09-ac7f-434f-9fa0-e1199b35ac79 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.901508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 989/1977] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:45:59.918272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 989/1978] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:45:59.919976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:45:59.931085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e45081-9cfa-4734-a488-a9c05084ce59 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:45:59.937535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:45:59.938421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:45:59.938701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:45:59.939381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] PATH_INFO: `/v3/projects/8f0ea03e15dc448786019de7a310433e/users/65e0b421e4654edabcd557145860be81/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:45:59.962876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0ea03e15dc448786019de7a310433e, user_id=65e0b421e4654edabcd557145860be81, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:45:59.967441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:45:59.988276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 990/1979] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:45:59 2022] PUT /identity/v3/projects/8f0ea03e15dc448786019de7a310433e/users/65e0b421e4654edabcd557145860be81/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:00.001155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.001744 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.001834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.023405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.025378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.043785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] MFA Rules not processed for user `65e0b421e4654edabcd557145860be81`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:00.047441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.120927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5088af1-85ce-4e96-ab53-6bb8e6e14334 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.136458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.137360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.137643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.137965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] PATH_INFO: `/v3/users/e9772d11a9f145f981b7664b83cb8d49` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.139282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9772d11a9f145f981b7664b83cb8d49)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.142185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.143338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 990/1980] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:00.148370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.163978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e765b-edb9-4797-b59d-d443dcacfe64 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.172035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.172874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.173844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.174329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.175880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.178248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.211455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9772d11a9f145f981b7664b83cb8d49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.218018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 991/1981] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:00.222607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.231140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9772d11a9f145f981b7664b83cb8d49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.233453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9772d11a9f145f981b7664b83cb8d49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.234160 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9772d11a9f145f981b7664b83cb8d49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.242234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9772d11a9f145f981b7664b83cb8d49'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.263026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 991/1982] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:00 2022] DELETE /identity/v3/users/e9772d11a9f145f981b7664b83cb8d49 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:00.264273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.266259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.325213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e97ca-26b2-4367-aeda-4076420d10f2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.364059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a0fd2-32ad-468a-91b2-31eee534ceb9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.369909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.371006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.371292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.371556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] PATH_INFO: `/v3/projects/68a2df7310e74b4496876dfe2812d5e1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.372840 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a2df7310e74b4496876dfe2812d5e1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.375000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.387208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef7ed1-f5d3-42c9-8382-2cd44baf70d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.387960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef7ed1-f5d3-42c9-8382-2cd44baf70d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.388275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef7ed1-f5d3-42c9-8382-2cd44baf70d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.388556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef7ed1-f5d3-42c9-8382-2cd44baf70d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.389278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef7ed1-f5d3-42c9-8382-2cd44baf70d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.391198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef7ed1-f5d3-42c9-8382-2cd44baf70d8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.417245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 992/1983] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:00.417245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088c18a-c9de-4e81-a2bb-adb355e156f8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.417245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088c18a-c9de-4e81-a2bb-adb355e156f8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.417245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088c18a-c9de-4e81-a2bb-adb355e156f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.430595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a2df7310e74b4496876dfe2812d5e1', 'request_id': 'req-7c137a58-e164-451a-a6b6-48d374054004'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.439409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a2df7310e74b4496876dfe2812d5e1', 'request_id': 'req-7c137a58-e164-451a-a6b6-48d374054004'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.439787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6088c18a-c9de-4e81-a2bb-adb355e156f8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:00.441866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6088c18a-c9de-4e81-a2bb-adb355e156f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.447179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a2df7310e74b4496876dfe2812d5e1', 'request_id': 'req-7c137a58-e164-451a-a6b6-48d374054004'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.467374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] The token cache is being invalidate because project 68a2df7310e74b4496876dfe2812d5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:00.467963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.468795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:00.471301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 992/1984] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:00 2022] DELETE /identity/v3/projects/68a2df7310e74b4496876dfe2812d5e1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:00.472211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.474010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.526205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 993/1985] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:00.534755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.535189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.535877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.557269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c137a58-e164-451a-a6b6-48d374054004 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.562257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:00.563428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.564103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.564416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.564693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.565458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.567173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.570185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.584591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 993/1986] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:00.584591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.588934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 994/1987] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:00.589380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.589772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.604577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0eaaf5-b210-4b31-8aea-dc9e5201a811 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.616436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965e5b-b402-4c35-9308-ad741b59c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.616436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965e5b-b402-4c35-9308-ad741b59c76f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.616436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965e5b-b402-4c35-9308-ad741b59c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.616436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965e5b-b402-4c35-9308-ad741b59c76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.616879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965e5b-b402-4c35-9308-ad741b59c76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.616879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965e5b-b402-4c35-9308-ad741b59c76f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.629064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 995/1988] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:00.630859 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.631152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.631430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.650569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:00.651804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.655294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e805a28-655f-43c8-992e-5967af2217fd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.659325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.660046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.660435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.660541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 996/1989] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:00.660897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.661304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.670854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.672223 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e4cc-b08f-4bf2-b22e-db7f1c1c2e92 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.672550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.676786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef801d82-dd06-4458-9d7b-9853d9116418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.678135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef801d82-dd06-4458-9d7b-9853d9116418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.678135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef801d82-dd06-4458-9d7b-9853d9116418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.678344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef801d82-dd06-4458-9d7b-9853d9116418 admin admin] PATH_INFO: `/v3/projects/167bb7ab9ba840778adaa492eb7034bc/users/72ba0610bbd242f8aa57e0b30a74794d/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.682894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 994/1990] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:00.683832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.698979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4ddf-7688-4f02-9cbd-d1116857a4e9 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.704279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef801d82-dd06-4458-9d7b-9853d9116418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=167bb7ab9ba840778adaa492eb7034bc, user_id=72ba0610bbd242f8aa57e0b30a74794d, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.707318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef801d82-dd06-4458-9d7b-9853d9116418 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.713320 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.714271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.714703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.714852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.716613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:00.719464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.721441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.740188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 995/1991] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:00.741745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.750983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 997/1992] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:00 2022] PUT /identity/v3/projects/167bb7ab9ba840778adaa492eb7034bc/users/72ba0610bbd242f8aa57e0b30a74794d/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:00.756398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e958f08-e7e6-4627-a350-a23478f504e0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.762346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.764439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.764439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.811569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] MFA Rules not processed for user `72ba0610bbd242f8aa57e0b30a74794d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:00.812853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:00.845938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.846617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.846994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.847277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.848440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.850607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:00.892364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 996/1993] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:00.895315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.915684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 998/1994] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:00.920542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:00.940414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63d709-c9d6-40fa-9011-ac0fca4dbdd6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.984832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532175f3-66b1-42d4-92ef-29c14cd170e0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:00.992489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:00.993263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:00.993613 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:00.993963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:00.994921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:00.998364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.010170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.012292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.012909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.013456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.014667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.017222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.024683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 997/1995] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:01.029773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.043904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb4ce0-8be1-42d2-99dd-bb86b982a0b9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.050940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.051133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 999/1996] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:00 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:01.052246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.052727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.053181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.054186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.055187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.056719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.069263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 998/1997] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:01.073150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.084694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4c08-f683-44e0-b91a-8521c8462e3d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.131017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c370b-b46f-44ea-aefc-9ad0eb9662ff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.137325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.138248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.138767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.139051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.139921 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.143444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.159290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.159980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.160366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.160647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.161399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.165535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.170765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1000/1998] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:01 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:01.176420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.188400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 999/1999] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:01.189635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.200924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d7fb0-ac6c-4ea1-bec6-1a35f6592b21 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.201484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02302bb9-1dcd-4651-894f-ea6308172b56 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.206672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.207793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.208130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.208253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.208621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] PATH_INFO: `/v3/projects/71ffa0e21b5d4e3c9cd8d32a670ab55d/users/1e3574fdcfa7420995da635879eb2dd1/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.208949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.209367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.209647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.210498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.213470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.225849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ffa0e21b5d4e3c9cd8d32a670ab55d, user_id=1e3574fdcfa7420995da635879eb2dd1, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.229249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1001/2000] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:01.230260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.233303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.234886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:01.255944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f72418-4bb5-4723-be51-c58762f483b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.261656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.262519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.263191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.263485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] PATH_INFO: `/v3/projects/fb6b3c3df57349d9b01cf6c84012e60d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.265122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6b3c3df57349d9b01cf6c84012e60d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.276690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.280255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1000/2001] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:01 2022] PUT /identity/v3/projects/71ffa0e21b5d4e3c9cd8d32a670ab55d/users/1e3574fdcfa7420995da635879eb2dd1/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:01.295006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.326114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6b3c3df57349d9b01cf6c84012e60d', 'request_id': 'req-a02b302a-1534-477b-aa59-a572e6ee0c2d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.337425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6b3c3df57349d9b01cf6c84012e60d', 'request_id': 'req-a02b302a-1534-477b-aa59-a572e6ee0c2d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.345238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6b3c3df57349d9b01cf6c84012e60d', 'request_id': 'req-a02b302a-1534-477b-aa59-a572e6ee0c2d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.360758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] The token cache is being invalidate because project fb6b3c3df57349d9b01cf6c84012e60d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:01.361045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.361713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a02b302a-1534-477b-aa59-a572e6ee0c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.363704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1002/2002] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:01 2022] DELETE /identity/v3/projects/fb6b3c3df57349d9b01cf6c84012e60d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:01.364981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.365263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.365547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.379054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a26d96-2d68-417c-9972-ea35fde41c0c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.393834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] MFA Rules not processed for user `1e3574fdcfa7420995da635879eb2dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:01.395055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:01.443252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.444281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.444569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.444896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] PATH_INFO: `/v3/projects/c96faae1e3374bb69115fa7b3a672007/users/216f32ed0f3f49a898150663ec21e887/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.461124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96faae1e3374bb69115fa7b3a672007, user_id=216f32ed0f3f49a898150663ec21e887, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.464121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.466586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1003/2003] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:01.467558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.469381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:01.481235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1001/2004] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:01 2022] PUT /identity/v3/projects/c96faae1e3374bb69115fa7b3a672007/users/216f32ed0f3f49a898150663ec21e887/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:01.482300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.484500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:01.490645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831d756-4b62-4611-a9f1-ba7718939842 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.558127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.558965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.559252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.559528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] PATH_INFO: `/v3/users/10312d841edd414fad7ecbdb51bd1383` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.560661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e36a81-0822-405d-8d1b-0fe0ba888b4c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.561893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] RBAC: Authorizing `identity:delete_user(user_id=10312d841edd414fad7ecbdb51bd1383)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.564257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.571375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.572039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.572359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.572629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.646779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10312d841edd414fad7ecbdb51bd1383'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.655750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10312d841edd414fad7ecbdb51bd1383'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.665888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10312d841edd414fad7ecbdb51bd1383'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.667957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10312d841edd414fad7ecbdb51bd1383'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.674569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.676040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.687028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10312d841edd414fad7ecbdb51bd1383'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:01.695566 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1004/2005] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:01 2022] DELETE /identity/v3/users/10312d841edd414fad7ecbdb51bd1383 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:01.696767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.698155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:01.713833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1002/2006] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:01.800510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ec32-0581-4458-a701-0b57c278c46a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.806901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617039f-d343-4da7-b293-2bb1a7faabb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.807939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617039f-d343-4da7-b293-2bb1a7faabb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.808480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617039f-d343-4da7-b293-2bb1a7faabb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.808915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617039f-d343-4da7-b293-2bb1a7faabb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.809866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617039f-d343-4da7-b293-2bb1a7faabb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.811677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617039f-d343-4da7-b293-2bb1a7faabb8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.822981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1005/2007] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:01.834344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:01.835752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:01.863697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651668e-418b-49db-a439-6dfacab96215 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:01.870749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:01.872006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.872565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.873065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] PATH_INFO: `/v3/projects/c96faae1e3374bb69115fa7b3a672007/users/216f32ed0f3f49a898150663ec21e887/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.877003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96faae1e3374bb69115fa7b3a672007, user_id=216f32ed0f3f49a898150663ec21e887, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:01.881468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:01.915497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1003/2008] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:01 2022] PUT /identity/v3/projects/c96faae1e3374bb69115fa7b3a672007/users/216f32ed0f3f49a898150663ec21e887/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:01.927728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:01.928144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:01.928455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:01.973334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] MFA Rules not processed for user `216f32ed0f3f49a898150663ec21e887`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:01.981348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.099645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1006/2009] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:02.119440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.129199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.130376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.131890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.276147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da66cd4-0df5-4a00-ac02-979f239c4a5a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.295587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65dd8b-b379-4137-b708-ea4a7d9627d3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.295817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a16156-6f8c-4cce-8b47-8d2a54c4aba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.295817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a16156-6f8c-4cce-8b47-8d2a54c4aba3 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.295817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a16156-6f8c-4cce-8b47-8d2a54c4aba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.295817 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a16156-6f8c-4cce-8b47-8d2a54c4aba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.302413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.303303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.303586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.303870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] PATH_INFO: `/v3/projects/f27d4c0a694246439d2ee73086b53005` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.306257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f27d4c0a694246439d2ee73086b53005)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.310475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.311522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a16156-6f8c-4cce-8b47-8d2a54c4aba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.320679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a16156-6f8c-4cce-8b47-8d2a54c4aba3 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.334458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1007/2010] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:02.381189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f27d4c0a694246439d2ee73086b53005', 'request_id': 'req-afc05401-f2dc-49dd-9394-f558773d14ce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:02.390426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f27d4c0a694246439d2ee73086b53005', 'request_id': 'req-afc05401-f2dc-49dd-9394-f558773d14ce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:02.396077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f27d4c0a694246439d2ee73086b53005', 'request_id': 'req-afc05401-f2dc-49dd-9394-f558773d14ce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:02.412073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] The token cache is being invalidate because project f27d4c0a694246439d2ee73086b53005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:02.412486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:02.413219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-afc05401-f2dc-49dd-9394-f558773d14ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:02.414817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1004/2011] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:02 2022] DELETE /identity/v3/projects/f27d4c0a694246439d2ee73086b53005 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:02.423070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.423070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.423070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.440726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:02.442774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.510072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1008/2012] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:02.519694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.519970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.520266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.539764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:02.541381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.549734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1005/2013] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:02.554357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.564507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d4ab5-3f39-443a-b90f-bbee7e3d1bc4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.569352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.570008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.570292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.570630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.571456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:02.571682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.573232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.589659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1009/2014] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:02.591424 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.600431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefad45d-c01b-44f6-9c19-3fe3bff6c91d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.605165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.605907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.606191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.606652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.608082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.610021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.643633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1006/2015] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:02 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:02.648023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.659100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25d72-781e-4ae0-9d30-4afb96cad2bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.738562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.738562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.738562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.738562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.739097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.739097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.759216 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1010/2016] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:02 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:02.764526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.782274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499afe-68c8-42a2-b59a-0fbce13efbe4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.787024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.787605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.787889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.788195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.788929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.790511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.802169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1007/2017] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:02.806336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.820145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f4b4e-484b-4a50-97ea-c3e773ccd875 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.821911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.822736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.823019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.823296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.824024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.825672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.837964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1011/2018] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:02.843232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.853851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0dc5-a9e2-4596-9e11-f2df0fab8ffe admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:02.859741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27beec-fb39-4a45-91f6-91525d0c9184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:02.862090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27beec-fb39-4a45-91f6-91525d0c9184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.862392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27beec-fb39-4a45-91f6-91525d0c9184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.862701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27beec-fb39-4a45-91f6-91525d0c9184 admin admin] PATH_INFO: `/v3/projects/9564780280f0425883d6d4b2eabb5991/users/c2eceeaba87d4fdab4963f35f5cd7552/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.878666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f27beec-fb39-4a45-91f6-91525d0c9184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9564780280f0425883d6d4b2eabb5991, user_id=c2eceeaba87d4fdab4963f35f5cd7552, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:02.881370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f27beec-fb39-4a45-91f6-91525d0c9184 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:02.881746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:02.883513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.899222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1008/2019] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:02 2022] PUT /identity/v3/projects/9564780280f0425883d6d4b2eabb5991/users/c2eceeaba87d4fdab4963f35f5cd7552/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:02.910324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:02.910619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:02.910895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:02.942163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] MFA Rules not processed for user `c2eceeaba87d4fdab4963f35f5cd7552`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:02.943362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:02.963677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061854d-2938-47c2-834a-6910490e0108 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:03.016319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1009/2020] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:03.019622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:03.020307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:03.020645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:03.020928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:03.022885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:03.027757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:03.029346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:03.040752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8150b5-ff9c-4af9-8c69-d3ff2fa01da4 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:03.045194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:03.045659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:03.045936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:03.046245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:03.054327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:03.055790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:03.065760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1010/2021] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:03.087804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:03.089398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0408d66-f1bb-4c76-9b6a-12cc5bbf6950 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:03.098145 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1012/2022] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:12.436187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:12.446129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4e81-53d2-4c90-9a37-4c62db5bf867 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:12.453992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:12.454639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:12.454918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:12.455191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:12.457005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:12.537175 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:12.539322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:12.552811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1011/2023] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:13.069842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b74e1-249d-4eb1-b8e4-3fa139d8cab8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:13.070411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b74e1-249d-4eb1-b8e4-3fa139d8cab8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:13.070882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b74e1-249d-4eb1-b8e4-3fa139d8cab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:13.090774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c10b74e1-249d-4eb1-b8e4-3fa139d8cab8 None None] MFA Rules not processed for user `9ad11a30da01449f82c1b94c7181860f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:13.092535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c10b74e1-249d-4eb1-b8e4-3fa139d8cab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:13.168841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1013/2024] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:14.947750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:14.951467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:15.030163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81965ff-4aa8-459b-bf45-b2c434e7b493 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:15.042323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:15.044959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:15.046701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:15.046701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] PATH_INFO: `/v3/users/65e0b421e4654edabcd557145860be81` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:15.050130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e0b421e4654edabcd557145860be81)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:15.059919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:15.136672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e0b421e4654edabcd557145860be81'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.155262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e0b421e4654edabcd557145860be81'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.156414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e0b421e4654edabcd557145860be81'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.157216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e0b421e4654edabcd557145860be81'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.165414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e0b421e4654edabcd557145860be81'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.201889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1012/2025] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:14 2022] DELETE /identity/v3/users/65e0b421e4654edabcd557145860be81 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:15.206272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:15.206715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:15.206832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:15.232855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:15.234344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:15.345455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1014/2026] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:15.355756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:15.357870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:15.452594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e49bf-6db0-42c2-a34c-6d4506a11cd9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:15.458778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:15.459553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:15.459965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:15.460416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:15.476468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:15.478057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:15.488679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1013/2027] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:15.766117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:15.767760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:15.795916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f30eb-b6cd-4eb9-b0b0-38c52f38e852 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:15.814274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:15.815594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:15.815879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:15.816150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] PATH_INFO: `/v3/projects/8f0ea03e15dc448786019de7a310433e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:15.817395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0ea03e15dc448786019de7a310433e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:15.819427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:15.904123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0ea03e15dc448786019de7a310433e', 'request_id': 'req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.912641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0ea03e15dc448786019de7a310433e', 'request_id': 'req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.918465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0ea03e15dc448786019de7a310433e', 'request_id': 'req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.931914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] The token cache is being invalidate because project 8f0ea03e15dc448786019de7a310433e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:15.932252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.932946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:15.937970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1015/2028] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:15 2022] DELETE /identity/v3/projects/8f0ea03e15dc448786019de7a310433e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:15.940135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:15.951409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166fb34-a9c1-4574-ab9c-c33c2b0ef267 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:16.019237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:16.020301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:16.020591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:16.020868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] PATH_INFO: `/v3/users/72ba0610bbd242f8aa57e0b30a74794d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:16.022094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ba0610bbd242f8aa57e0b30a74794d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:16.024006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:16.083441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ba0610bbd242f8aa57e0b30a74794d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.096350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ba0610bbd242f8aa57e0b30a74794d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.097055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ba0610bbd242f8aa57e0b30a74794d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.097655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ba0610bbd242f8aa57e0b30a74794d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.104102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ba0610bbd242f8aa57e0b30a74794d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.117370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1014/2029] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:15 2022] DELETE /identity/v3/users/72ba0610bbd242f8aa57e0b30a74794d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:16.399989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:16.401760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:16.415540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab7945-80d9-4c4f-b7ef-75a87cc4c351 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:16.484370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:16.485243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:16.485524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:16.485795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] PATH_INFO: `/v3/projects/167bb7ab9ba840778adaa492eb7034bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:16.486999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] RBAC: Authorizing `identity:delete_project(project_id=167bb7ab9ba840778adaa492eb7034bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:16.488944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:16.543115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '167bb7ab9ba840778adaa492eb7034bc', 'request_id': 'req-4c4ef10e-c757-418e-a20c-3e52dc76817a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.550823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '167bb7ab9ba840778adaa492eb7034bc', 'request_id': 'req-4c4ef10e-c757-418e-a20c-3e52dc76817a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.557831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '167bb7ab9ba840778adaa492eb7034bc', 'request_id': 'req-4c4ef10e-c757-418e-a20c-3e52dc76817a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.579258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] The token cache is being invalidate because project 167bb7ab9ba840778adaa492eb7034bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:16.579683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.583835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.585675 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1016/2030] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:16 2022] DELETE /identity/v3/projects/167bb7ab9ba840778adaa492eb7034bc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:16.589757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:16.593803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:16.608215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8fdae-d2ba-489d-bf6a-67203cd599b0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:16.681865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:16.683606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:16.684090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:16.685433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] PATH_INFO: `/v3/users/216f32ed0f3f49a898150663ec21e887` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:16.687196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216f32ed0f3f49a898150663ec21e887)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:16.689815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:16.750621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216f32ed0f3f49a898150663ec21e887'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.765373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216f32ed0f3f49a898150663ec21e887'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.766231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216f32ed0f3f49a898150663ec21e887'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.766959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216f32ed0f3f49a898150663ec21e887'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.774000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216f32ed0f3f49a898150663ec21e887'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:16.792825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1015/2031] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:16 2022] DELETE /identity/v3/users/216f32ed0f3f49a898150663ec21e887 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:17.390561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.392182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.406044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ef10e-c757-418e-a20c-3e52dc76817a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.478218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.478697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.479583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.479870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.480225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] PATH_INFO: `/v3/users/b00b6242e52a4ab1bc98280fe7242647` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.481167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.481718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00b6242e52a4ab1bc98280fe7242647)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.484323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.497502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1804a3-3eae-4c2e-a590-e3f8e6595c80 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.516541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.518163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.519187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.519727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] PATH_INFO: `/v3/projects/c96faae1e3374bb69115fa7b3a672007` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.521146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96faae1e3374bb69115fa7b3a672007)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.523953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.556698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00b6242e52a4ab1bc98280fe7242647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.564860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00b6242e52a4ab1bc98280fe7242647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.574177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00b6242e52a4ab1bc98280fe7242647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.574879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00b6242e52a4ab1bc98280fe7242647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.579293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96faae1e3374bb69115fa7b3a672007', 'request_id': 'req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.587319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96faae1e3374bb69115fa7b3a672007', 'request_id': 'req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.588644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bbf476bb-ff66-4f2e-9b22-4401007a1c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00b6242e52a4ab1bc98280fe7242647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.593657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96faae1e3374bb69115fa7b3a672007', 'request_id': 'req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.595234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1017/2032] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:17 2022] DELETE /identity/v3/users/b00b6242e52a4ab1bc98280fe7242647 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:17.605080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] The token cache is being invalidate because project c96faae1e3374bb69115fa7b3a672007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:17.605372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.606012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2ca291d4-ef79-4c76-ad0a-0984d9bb0a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.607851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1016/2033] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:17 2022] DELETE /identity/v3/projects/c96faae1e3374bb69115fa7b3a672007 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:17.614596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.614885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.615164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.636712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:17.639106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.696635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1018/2034] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:17.705827 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.706112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.706417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.729288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:17.730489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.739028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1017/2035] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:17.743555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.753858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89833edb-82e6-4ea1-a529-03fd1fb24ab7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.759684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.760668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.761086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.761393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.762567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:17.762953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.764546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.769346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.771092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.784007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea1cb2-e4ef-4312-a7ef-30b1958a38f3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.788275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1019/2036] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:46:17.789900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.790855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.791246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.791472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.791857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] PATH_INFO: `/v3/projects/4a12890be016462ea430af4ff0080553` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.793273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a12890be016462ea430af4ff0080553)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.802863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.803468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa680a5-5ea5-4467-aae9-b974f7d08e08 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.807976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.808659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.808936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.809205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.810294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.812153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.838955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1020/2037] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:17.843618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.850479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a12890be016462ea430af4ff0080553', 'request_id': 'req-a95e656d-2c47-4350-9759-805b0d6d9de1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.859643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a12890be016462ea430af4ff0080553', 'request_id': 'req-a95e656d-2c47-4350-9759-805b0d6d9de1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.865210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a12890be016462ea430af4ff0080553', 'request_id': 'req-a95e656d-2c47-4350-9759-805b0d6d9de1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.876154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] The token cache is being invalidate because project 4a12890be016462ea430af4ff0080553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:17.876467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.877121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a95e656d-2c47-4350-9759-805b0d6d9de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:17.878776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1018/2038] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:17 2022] DELETE /identity/v3/projects/4a12890be016462ea430af4ff0080553 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:17.883542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f399f33-75e0-4cd3-8322-665b0faed97e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.883831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f399f33-75e0-4cd3-8322-665b0faed97e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.884108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f399f33-75e0-4cd3-8322-665b0faed97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.902450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4f399f33-75e0-4cd3-8322-665b0faed97e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:17.904348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4f399f33-75e0-4cd3-8322-665b0faed97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.908410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328c45-81f1-4e01-8157-99fc20436060 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.912569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1019/2039] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:17.913773 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.914440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.914716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.914985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.915664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.917626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.920867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.921158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.921436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.940876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:17.942228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.944076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1021/2040] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:17.948015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.949601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:17.953851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1020/2041] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:17.957721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.961424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d79a0f-5542-4419-851c-6e82a4ef6127 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.966080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.966443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6a685-837b-40af-ab72-a3ca2b3b7874 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.966723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.967048 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.967267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.967990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.969597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.971027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.971665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.971951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:17.972262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:17.972984 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:17.973409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:17.974979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:17.979517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1022/2042] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:17.984048 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.990718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1021/2043] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:46:17.991831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:17.993467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16343c57-18be-4128-821d-604e94fadde2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:17.998711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:17.999625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:17.999905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.000206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.000922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.002041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea30b91-fa44-4aee-b8d0-41319360d558 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.002783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.006593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.007218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.007501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.007774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.008487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.010342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.013564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1023/2044] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:18.017493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.026965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b48483-1850-4b4d-acc1-c3e7529e3202 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.032621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22486eee-ebd8-403c-83ed-7a8cc1f10e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.033561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486eee-ebd8-403c-83ed-7a8cc1f10e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.033908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486eee-ebd8-403c-83ed-7a8cc1f10e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.034393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22486eee-ebd8-403c-83ed-7a8cc1f10e3b admin admin] PATH_INFO: `/v3/projects/c14393e016e7466fa001290a8ba91c1a/users/9a6a690b3d9c448ba9ced3b7486bf795/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.039884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1022/2045] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:17 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:18.040883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.049456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22486eee-ebd8-403c-83ed-7a8cc1f10e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14393e016e7466fa001290a8ba91c1a, user_id=9a6a690b3d9c448ba9ced3b7486bf795, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.052079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22486eee-ebd8-403c-83ed-7a8cc1f10e3b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.064463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1024/2046] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:18 2022] PUT /identity/v3/projects/c14393e016e7466fa001290a8ba91c1a/users/9a6a690b3d9c448ba9ced3b7486bf795/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:18.076028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.076430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.076702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.100495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88755f-1328-48eb-ae7b-f72a59355870 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.102534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] MFA Rules not processed for user `9a6a690b3d9c448ba9ced3b7486bf795`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:18.103722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.154761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.155420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.155699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.155970 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.156687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.158617 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.166131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1025/2047] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:18.178562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.179293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1023/2048] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:18 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:18.179955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.183328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.192573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de24cd-bde8-4ee3-b52f-00158100d96b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.197145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.197797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.198083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.198360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.199079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.201352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.211047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1024/2049] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:18.214981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.224891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c8d01-a8c6-44ea-9c75-9af27857a01c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.229424 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.230057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.230338 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.230618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.231413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.233210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.235519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017848ca-4a48-41e3-8cc0-b8912bb9ce20 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.239376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.240172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.240483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.240812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.244456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1025/2050] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:18.249497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.250256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.251681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.258799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faaa0e-adeb-4ded-b17c-090662c73893 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.260418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1026/2051] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:18.265738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c66c-8bd0-4acf-8170-66e3fae70697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.266623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c66c-8bd0-4acf-8170-66e3fae70697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.266902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c66c-8bd0-4acf-8170-66e3fae70697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.267173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c66c-8bd0-4acf-8170-66e3fae70697 admin admin] PATH_INFO: `/v3/projects/e25f8b8d3790465a8f626d2218d89c42/users/b0a3347ce7fe40d9a60849b50b1bed53/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.280190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.282293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27c66c-8bd0-4acf-8170-66e3fae70697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e25f8b8d3790465a8f626d2218d89c42, user_id=b0a3347ce7fe40d9a60849b50b1bed53, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.284959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27c66c-8bd0-4acf-8170-66e3fae70697 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.290998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb301af-2b92-4c8b-86fe-c782756ca700 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.297732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1026/2052] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:18 2022] PUT /identity/v3/projects/e25f8b8d3790465a8f626d2218d89c42/users/b0a3347ce7fe40d9a60849b50b1bed53/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:18.308994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.309908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.310630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.336574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] MFA Rules not processed for user `b0a3347ce7fe40d9a60849b50b1bed53`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:18.337805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.347141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.348022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.348329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.348600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] PATH_INFO: `/v3/users/9a6a690b3d9c448ba9ced3b7486bf795` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.349788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6a690b3d9c448ba9ced3b7486bf795)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.351684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.400068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6a690b3d9c448ba9ced3b7486bf795'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:18.401336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1027/2053] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:18.409814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6a690b3d9c448ba9ced3b7486bf795'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:18.412317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.413675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.417507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6a690b3d9c448ba9ced3b7486bf795'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:18.418141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6a690b3d9c448ba9ced3b7486bf795'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:18.429753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ce1b739d-b9f6-4b25-be5c-a6e7e1fa910b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6a690b3d9c448ba9ced3b7486bf795'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:18.436076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1027/2054] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:18 2022] DELETE /identity/v3/users/9a6a690b3d9c448ba9ced3b7486bf795 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:18.440453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.440750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.441033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.458733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:18.459965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.469689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d816ff8-010b-4da7-9a2f-dcf38bdc4871 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.520742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.521543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.522000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.522381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.524631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.525495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1028/2055] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:18.535285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:18.537143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:18.610389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.617749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.625704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77cc8e-17be-46c7-a5cf-f03888aaec85 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:18.630000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1028/2056] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:18.638652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60291555-ba31-45dc-9fee-47d5dacb44f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:18.639314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60291555-ba31-45dc-9fee-47d5dacb44f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:18.639729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60291555-ba31-45dc-9fee-47d5dacb44f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:18.639858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60291555-ba31-45dc-9fee-47d5dacb44f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:18.650914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60291555-ba31-45dc-9fee-47d5dacb44f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:18.652372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60291555-ba31-45dc-9fee-47d5dacb44f5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:18.667894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1029/2057] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:19.189823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.191271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:19.205788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d440f1e-e84a-46cc-b37c-f6f4fac5b87a service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.210961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.211825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.212107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.212406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] PATH_INFO: `/v3/projects/c14393e016e7466fa001290a8ba91c1a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.213596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14393e016e7466fa001290a8ba91c1a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.216333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.260354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14393e016e7466fa001290a8ba91c1a', 'request_id': 'req-9b4d110a-34e9-45c0-8e35-603058c410c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:19.268110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14393e016e7466fa001290a8ba91c1a', 'request_id': 'req-9b4d110a-34e9-45c0-8e35-603058c410c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:19.273709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14393e016e7466fa001290a8ba91c1a', 'request_id': 'req-9b4d110a-34e9-45c0-8e35-603058c410c0'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:19.284573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] The token cache is being invalidate because project c14393e016e7466fa001290a8ba91c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:19.284863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:19.285507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-9b4d110a-34e9-45c0-8e35-603058c410c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:19.287053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1029/2058] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:19 2022] DELETE /identity/v3/projects/c14393e016e7466fa001290a8ba91c1a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:19.291989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.292317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.292596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.311449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:19.312711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:19.369210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1030/2059] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:19.378758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.379044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.379318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.397615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:19.398799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:19.407071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1030/2060] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:19.411128 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.420492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6d0a9-ecca-434d-af99-b4ee528ad704 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.425127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.425787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.426071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.426344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.427068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:19.427378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.429032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.440516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1031/2061] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:19.444610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.454752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e8bb1-09ae-459b-9d80-17b2d20a0d15 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.459344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.459979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.460295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.460599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.461313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.463189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.488255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1031/2062] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:19.492540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.502559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740cd45-df58-455e-98e0-a882f034a74a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.556215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.556805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.557087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.557362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.558046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.560148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.580453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1032/2063] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:19.584328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.595232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873912a5-b4e3-4a27-a74c-48cdfda4e924 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.599813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.600515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.600788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.601066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.601795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.603656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.614360 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1032/2064] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:19.618463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.628863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2b05-5358-4596-85be-ea110cdfaabe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.633619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042863c-0cd4-4577-b45c-c118308fc4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.634262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042863c-0cd4-4577-b45c-c118308fc4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.634539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042863c-0cd4-4577-b45c-c118308fc4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.634829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042863c-0cd4-4577-b45c-c118308fc4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.635559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042863c-0cd4-4577-b45c-c118308fc4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.637176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042863c-0cd4-4577-b45c-c118308fc4b1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.647759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1033/2065] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:19.651563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.661820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0f444-73d8-4f58-834b-5d8a14d48886 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.666303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.667157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.667433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.667704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] PATH_INFO: `/v3/projects/23a16f3f21b14bf0842a0a53ddb686c1/users/e474cf7a48e540b0a42c2a28c37e8958/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.682350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a16f3f21b14bf0842a0a53ddb686c1, user_id=e474cf7a48e540b0a42c2a28c37e8958, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.685082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.697922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1033/2066] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:19 2022] PUT /identity/v3/projects/23a16f3f21b14bf0842a0a53ddb686c1/users/e474cf7a48e540b0a42c2a28c37e8958/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:19.708670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.708966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.709250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.737483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] MFA Rules not processed for user `e474cf7a48e540b0a42c2a28c37e8958`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:19.738798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:19.811182 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1034/2067] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:19.815586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.887602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7952-ac67-47be-ab2a-c49b046c1e09 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:19.892293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:19.892953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:19.893847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:19.894151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:19.894933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:19.897156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:19.924745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1034/2068] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:19.928903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:19.939825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e31fed-7dd6-4f45-bb26-f8778e09ba53 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.002224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.002898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.003187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.003560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.004150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.006107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.029549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1035/2069] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:19 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:20.030962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.041376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e7a8b-2850-4058-bba8-2884ede33f8e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.047230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.047888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.048192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.048486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.049217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.050822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.062663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1035/2070] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:20.066379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.076267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b30a46-c735-4c2b-9c38-dee48a68c8c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.081147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b9539-55a0-46bc-bbe8-cc18f8bbf47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.081789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b9539-55a0-46bc-bbe8-cc18f8bbf47d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.082075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b9539-55a0-46bc-bbe8-cc18f8bbf47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.082352 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b9539-55a0-46bc-bbe8-cc18f8bbf47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.083075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b9539-55a0-46bc-bbe8-cc18f8bbf47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.084699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b9539-55a0-46bc-bbe8-cc18f8bbf47d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.094836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1036/2071] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:20.098849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.109337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575df89-2bfb-4772-88cd-6b591e29314b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.113853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.114711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.114991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.115262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] PATH_INFO: `/v3/projects/1820c296693b43e49bb2b4fd2cd8c8b7/users/b9cfe3a3807f4497b8596682605e81f7/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.129603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1820c296693b43e49bb2b4fd2cd8c8b7, user_id=b9cfe3a3807f4497b8596682605e81f7, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.132239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.151361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1036/2072] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:20 2022] PUT /identity/v3/projects/1820c296693b43e49bb2b4fd2cd8c8b7/users/b9cfe3a3807f4497b8596682605e81f7/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:20.162918 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.163211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.163490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.189424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] MFA Rules not processed for user `b9cfe3a3807f4497b8596682605e81f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:20.190618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:20.251827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1037/2073] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:20.256236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.316946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a1d4-a873-41e4-8776-bb29df85b84e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.321473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.322126 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.322412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.322693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.323387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.325848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.350332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1037/2074] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:20 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:20.354389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.364958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b41b2-5e5e-4965-9973-d7140cf2ec5c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.419026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.419694 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.419978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.420292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.420981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.422892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.444787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1038/2075] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:20 2022] POST /identity/v3/users => generated 298 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:20.445940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.456635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bea4d7-362b-468f-a844-4556456db043 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.461200 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.461894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.462177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.462451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.463170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.464784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.475438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1038/2076] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:20.479458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.489365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd274981-ade9-431b-8711-f564692cdbd9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.493964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.494823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.495100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.495369 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] PATH_INFO: `/v3/projects/c821db2fe1fd43d7b3cc95756b0678a5/users/10f82c7abae64724b2844f5baa3482fb/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.510082 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c821db2fe1fd43d7b3cc95756b0678a5, user_id=10f82c7abae64724b2844f5baa3482fb, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.513389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.525345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1039/2077] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:20 2022] PUT /identity/v3/projects/c821db2fe1fd43d7b3cc95756b0678a5/users/10f82c7abae64724b2844f5baa3482fb/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:20.529817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.539442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57ccb-8f9e-4663-aa2a-dc7800a214ff admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.595442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e851824-4136-4756-bc63-e86d22f45c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.596151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e851824-4136-4756-bc63-e86d22f45c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.596373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e851824-4136-4756-bc63-e86d22f45c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.596645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e851824-4136-4756-bc63-e86d22f45c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.597372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e851824-4136-4756-bc63-e86d22f45c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.598985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e851824-4136-4756-bc63-e86d22f45c2d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.609951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1039/2078] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:20.614097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.628997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7526ce-a648-48b0-9cba-38e2964eb094 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.635472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.636860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.637276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.637695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] PATH_INFO: `/v3/projects/c821db2fe1fd43d7b3cc95756b0678a5/users/10f82c7abae64724b2844f5baa3482fb/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.640586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c821db2fe1fd43d7b3cc95756b0678a5, user_id=10f82c7abae64724b2844f5baa3482fb, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.644935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.657061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1040/2079] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:20 2022] PUT /identity/v3/projects/c821db2fe1fd43d7b3cc95756b0678a5/users/10f82c7abae64724b2844f5baa3482fb/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:20.669806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe7936-921a-4255-b809-1331530b146d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.670157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe7936-921a-4255-b809-1331530b146d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.670548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe7936-921a-4255-b809-1331530b146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.699645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f0fe7936-921a-4255-b809-1331530b146d None None] MFA Rules not processed for user `10f82c7abae64724b2844f5baa3482fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:20.701637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f0fe7936-921a-4255-b809-1331530b146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:20.766669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1040/2080] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:20.777905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:20.780257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:20.834560 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e4d6-f372-4b0b-bee4-026380a78f53 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:20.838139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:20.838766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:20.839045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:20.839320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:20.894848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:20.896529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:20.907273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1041/2081] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:23.598519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe7936-921a-4255-b809-1331530b146d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:23.599921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f0fe7936-921a-4255-b809-1331530b146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:23.614520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe7936-921a-4255-b809-1331530b146d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:23.624975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:23.625808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:23.626094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:23.626465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:23.688731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:23.690304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:23.699644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1041/2082] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:24.346236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:24.354633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72737d5b-61bc-4f84-9409-c122d2d9cd58 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:24.359084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24150e-a6b4-4519-9426-6bb53e5d9556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:24.359740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24150e-a6b4-4519-9426-6bb53e5d9556 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:24.360015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24150e-a6b4-4519-9426-6bb53e5d9556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:24.360316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24150e-a6b4-4519-9426-6bb53e5d9556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:24.370453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24150e-a6b4-4519-9426-6bb53e5d9556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:24.373470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24150e-a6b4-4519-9426-6bb53e5d9556 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:24.388607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1042/2083] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:27.217119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:27.219565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:27.317193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe0e35-31ba-4cc1-befd-84552bb47a84 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:27.321645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:27.322496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:27.322773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:27.323042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] PATH_INFO: `/v3/users/1e3574fdcfa7420995da635879eb2dd1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:27.324225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3574fdcfa7420995da635879eb2dd1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:27.326115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:27.372118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3574fdcfa7420995da635879eb2dd1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.384365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3574fdcfa7420995da635879eb2dd1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.385036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3574fdcfa7420995da635879eb2dd1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.385638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3574fdcfa7420995da635879eb2dd1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.393503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3574fdcfa7420995da635879eb2dd1'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.406085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1042/2084] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:27 2022] DELETE /identity/v3/users/1e3574fdcfa7420995da635879eb2dd1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:27.411185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100b611-e648-4c0d-83ef-152077731085 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:27.411797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100b611-e648-4c0d-83ef-152077731085 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:27.411797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100b611-e648-4c0d-83ef-152077731085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:27.429255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a100b611-e648-4c0d-83ef-152077731085 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:27.430453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a100b611-e648-4c0d-83ef-152077731085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:27.491675 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1043/2085] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:27.501733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:27.503805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:27.559387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a37ef-3fe4-4771-92e8-fbd9c7b46c6d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:27.563369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf19e96-aa3a-41f0-8fd0-84c4c69df9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:27.564130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf19e96-aa3a-41f0-8fd0-84c4c69df9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:27.564454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf19e96-aa3a-41f0-8fd0-84c4c69df9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:27.564731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf19e96-aa3a-41f0-8fd0-84c4c69df9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:27.573771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf19e96-aa3a-41f0-8fd0-84c4c69df9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:27.575260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf19e96-aa3a-41f0-8fd0-84c4c69df9c9 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:27.585812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1043/2086] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:27.857762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100b611-e648-4c0d-83ef-152077731085 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:27.859378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a100b611-e648-4c0d-83ef-152077731085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:27.878240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100b611-e648-4c0d-83ef-152077731085 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:27.884307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:27.885274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:27.885557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:27.885828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] PATH_INFO: `/v3/projects/71ffa0e21b5d4e3c9cd8d32a670ab55d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:27.887004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ffa0e21b5d4e3c9cd8d32a670ab55d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:27.888910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:27.938135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ffa0e21b5d4e3c9cd8d32a670ab55d', 'request_id': 'req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.944295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ffa0e21b5d4e3c9cd8d32a670ab55d', 'request_id': 'req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.949642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ffa0e21b5d4e3c9cd8d32a670ab55d', 'request_id': 'req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.962990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] The token cache is being invalidate because project 71ffa0e21b5d4e3c9cd8d32a670ab55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:27.963284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.963896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-669997bc-ed8b-43bc-8e21-ce30b58ca8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:27.965462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1044/2087] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:27 2022] DELETE /identity/v3/projects/71ffa0e21b5d4e3c9cd8d32a670ab55d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:27.970803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:27.971092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:27.971368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:27.989942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:27.991201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.061128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1044/2088] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:28.064442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.065116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.065429 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.089262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:28.090799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.101592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1045/2089] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:28.106161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.119339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea03c81-1d69-4a91-b487-cdc4fa1d315d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.127312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.128261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.128753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.129201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.130498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:28.130998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.133800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.148744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1045/2090] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:28.154594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.169504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058070f8-1a6a-46c1-8345-3c353b292118 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.178012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.178702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.178986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.179258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.179961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.181973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.229630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1046/2091] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:28 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:28.235640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.254524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d8f5-c697-406d-8a95-3689b64ad701 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.327555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.328358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.328719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.329188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.329880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.331799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.360215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1046/2092] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:28 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:28.361632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.361863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.363400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.372906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87407-50d2-41d1-a7e1-d7edb20c0708 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.378026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c1e68-51cf-405d-9fe8-2207134d0b8c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.380257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.381134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.381424 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.381712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.382586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.384366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.385079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.386189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.386538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.386881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] PATH_INFO: `/v3/users/e474cf7a48e540b0a42c2a28c37e8958` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.388183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e474cf7a48e540b0a42c2a28c37e8958)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.390903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.397317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1047/2093] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:28.400937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.412556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35df012-4c47-4d3b-95f3-a41e826b3c3e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.418872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.419567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.419854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.420136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.422793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.425076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.438278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1048/2094] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:28.442919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.455496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e474cf7a48e540b0a42c2a28c37e8958'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:28.456105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c0218-e128-4d28-8bb5-175af1860af0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.461038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952211a-9620-451e-beb6-c432ed704227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.462029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952211a-9620-451e-beb6-c432ed704227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.462431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952211a-9620-451e-beb6-c432ed704227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.462793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952211a-9620-451e-beb6-c432ed704227 admin admin] PATH_INFO: `/v3/projects/e53ce05ac35c48aeb17842025bb4c455/users/2e95c3d471634389a70bfcd7515085ae/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.463390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e474cf7a48e540b0a42c2a28c37e8958'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:28.475350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e474cf7a48e540b0a42c2a28c37e8958'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:28.476046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e474cf7a48e540b0a42c2a28c37e8958'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:28.480256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952211a-9620-451e-beb6-c432ed704227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53ce05ac35c48aeb17842025bb4c455, user_id=2e95c3d471634389a70bfcd7515085ae, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.482947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952211a-9620-451e-beb6-c432ed704227 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.492101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c42377e-307d-4fe1-9787-0dba04f8296a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e474cf7a48e540b0a42c2a28c37e8958'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:28.500658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1049/2096] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:28 2022] PUT /identity/v3/projects/e53ce05ac35c48aeb17842025bb4c455/users/2e95c3d471634389a70bfcd7515085ae/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:28.504244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1047/2096] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:28 2022] DELETE /identity/v3/users/e474cf7a48e540b0a42c2a28c37e8958 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:28.505024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.505345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.505678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.516196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.516614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.516886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.535841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:28.537433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.550071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] MFA Rules not processed for user `2e95c3d471634389a70bfcd7515085ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:28.551671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.622994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1050/2097] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:28.634929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.636385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.645910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1048/2098] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:28.659385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:28.661029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:28.698741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602706c1-5d1b-4c0f-899e-851f4c79e2cc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.702508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.703162 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.703430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.703702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.712760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.714177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.718338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902952e-aa4c-4ed0-b096-4ae59682ebf7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:28.721944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:28.722596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:28.722870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:28.723139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:28.724103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1051/2099] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:28.731367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:28.732809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:28.742179 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1049/2100] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:29.049558 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:29.051263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:29.075556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70a36-47a6-4e7f-b68e-5263756fe43f service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:29.085015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:29.086046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:29.086372 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:29.086698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] PATH_INFO: `/v3/projects/23a16f3f21b14bf0842a0a53ddb686c1` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:29.088231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a16f3f21b14bf0842a0a53ddb686c1)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:29.090398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:29.142267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a16f3f21b14bf0842a0a53ddb686c1', 'request_id': 'req-5184a167-9e62-43c9-b866-06536b29f877'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.150425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a16f3f21b14bf0842a0a53ddb686c1', 'request_id': 'req-5184a167-9e62-43c9-b866-06536b29f877'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.156830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a16f3f21b14bf0842a0a53ddb686c1', 'request_id': 'req-5184a167-9e62-43c9-b866-06536b29f877'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.167600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] The token cache is being invalidate because project 23a16f3f21b14bf0842a0a53ddb686c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:29.167882 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.168541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.170092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1052/2101] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:29 2022] DELETE /identity/v3/projects/23a16f3f21b14bf0842a0a53ddb686c1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:29.174908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:29.185347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ff634-fdf6-450f-bf24-a421f64dcc01 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:29.242014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:29.242884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:29.243166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:29.243438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] PATH_INFO: `/v3/users/b9cfe3a3807f4497b8596682605e81f7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:29.244643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9cfe3a3807f4497b8596682605e81f7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:29.246559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:29.294977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9cfe3a3807f4497b8596682605e81f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.305692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9cfe3a3807f4497b8596682605e81f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.314042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9cfe3a3807f4497b8596682605e81f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.314805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9cfe3a3807f4497b8596682605e81f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.327424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9cfe3a3807f4497b8596682605e81f7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.333579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1050/2102] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:29 2022] DELETE /identity/v3/users/b9cfe3a3807f4497b8596682605e81f7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:29.677829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:29.679437 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:29.693349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184a167-9e62-43c9-b866-06536b29f877 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:29.768498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:29.769456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:29.769950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:29.770229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] PATH_INFO: `/v3/projects/1820c296693b43e49bb2b4fd2cd8c8b7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:29.771941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1820c296693b43e49bb2b4fd2cd8c8b7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:29.773936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:29.829210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1820c296693b43e49bb2b4fd2cd8c8b7', 'request_id': 'req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.836334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1820c296693b43e49bb2b4fd2cd8c8b7', 'request_id': 'req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.841692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1820c296693b43e49bb2b4fd2cd8c8b7', 'request_id': 'req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.853022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] The token cache is being invalidate because project 1820c296693b43e49bb2b4fd2cd8c8b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:29.853310 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.853937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-1fadf53a-a48d-4d63-9c70-63368ed8e8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.855478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1053/2103] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:29 2022] DELETE /identity/v3/projects/1820c296693b43e49bb2b4fd2cd8c8b7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:29.860605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:29.862137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:29.875818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb115-1a28-463b-be7d-0ab95614a0dc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:29.935522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:29.935818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:29.936543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:29.936826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] PATH_INFO: `/v3/users/10f82c7abae64724b2844f5baa3482fb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:29.938016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f82c7abae64724b2844f5baa3482fb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:29.939966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:29.984913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f82c7abae64724b2844f5baa3482fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:29.991705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f82c7abae64724b2844f5baa3482fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.000555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f82c7abae64724b2844f5baa3482fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.001149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f82c7abae64724b2844f5baa3482fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.013140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f82c7abae64724b2844f5baa3482fb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.019584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1051/2104] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:29 2022] DELETE /identity/v3/users/10f82c7abae64724b2844f5baa3482fb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:30.134298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.134585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.134863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.180571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] MFA Rules not processed for user `2e95c3d471634389a70bfcd7515085ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:30.181876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.255862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1054/2105] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:30.341094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:30.342770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.407891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:30.415433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.431132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccf305-ccf4-49d3-84cd-a820ab1dba5b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:30.435995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:30.436923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.437205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.437483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] PATH_INFO: `/v3/projects/c821db2fe1fd43d7b3cc95756b0678a5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.438698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c821db2fe1fd43d7b3cc95756b0678a5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:30.440946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:30.517475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb69ef9-1714-49c5-908a-4a4259d410cf None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:30.526658 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c821db2fe1fd43d7b3cc95756b0678a5', 'request_id': 'req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.535533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c821db2fe1fd43d7b3cc95756b0678a5', 'request_id': 'req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.544110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c821db2fe1fd43d7b3cc95756b0678a5', 'request_id': 'req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.561783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] The token cache is being invalidate because project c821db2fe1fd43d7b3cc95756b0678a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:30.562698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.564153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee88b0bc-dafe-421b-9b25-bbcd4e14f140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.569220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1052/2106] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:30 2022] DELETE /identity/v3/projects/c821db2fe1fd43d7b3cc95756b0678a5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:30.571792 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206d3fc-c376-44f3-afff-a8e0866da467 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.572363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206d3fc-c376-44f3-afff-a8e0866da467 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.572593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206d3fc-c376-44f3-afff-a8e0866da467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.601446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a206d3fc-c376-44f3-afff-a8e0866da467 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:30.602645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a206d3fc-c376-44f3-afff-a8e0866da467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.648383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:30.648969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.649273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.650411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] PATH_INFO: `/v3/users/b0a3347ce7fe40d9a60849b50b1bed53` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.652419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a3347ce7fe40d9a60849b50b1bed53)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:30.654624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:30.684591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1053/2107] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:30.693291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.693782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.694688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.721018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:30.722890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.731618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1054/2108] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:30.732572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a3347ce7fe40d9a60849b50b1bed53'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.735740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:30.745993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d5677-0ff3-43df-b0cf-abc359576f8c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:30.746524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a3347ce7fe40d9a60849b50b1bed53'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.747465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a3347ce7fe40d9a60849b50b1bed53'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.748394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a3347ce7fe40d9a60849b50b1bed53'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.755896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:30.756764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.757194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.757720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.757990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a3347ce7fe40d9a60849b50b1bed53'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:30.758672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:30.759083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:30.760877 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b530b5-50a2-48ac-b706-d734f5dc99e8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:30.775009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1055/2109] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:30 2022] DELETE /identity/v3/users/b0a3347ce7fe40d9a60849b50b1bed53 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:30.779044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1055/2110] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:46:30.780958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:30.781420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.781800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.782177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.783105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.808405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:30.810042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.919350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcc0b9-202c-46f1-9a6a-9d623ce1fb3a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:30.929886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:30.930791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:30.932211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:30.934295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:30.943197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1056/2111] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:30.953237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:30.955164 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:30.959997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:30.961721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:30.991570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1056/2112] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:30 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:30.992673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.067739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93eb26-4c3b-42ec-976d-cc456505b6f3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.075889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa9fd2-618a-4909-87db-5faae1e99c6c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.082172 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.083376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.083658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.083933 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.084662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.086554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.121021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1057/2113] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:30 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:31.122428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.134475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf733a-78cb-4284-b601-d0c2d7ad8d97 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.139205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.140635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.140992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.141336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.142322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.144543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.151497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.152206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.152502 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.152777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.164140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.165010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1058/2114] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:31.168112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.173181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.183537 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1057/2115] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:31.186170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b45ae3-6c37-422b-8d68-f7fb951d112c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.191192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8aca9-088a-4bed-8cf6-26eba2b7d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.191870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8aca9-088a-4bed-8cf6-26eba2b7d571 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.192207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8aca9-088a-4bed-8cf6-26eba2b7d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.192500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8aca9-088a-4bed-8cf6-26eba2b7d571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.193234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8aca9-088a-4bed-8cf6-26eba2b7d571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.194879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8aca9-088a-4bed-8cf6-26eba2b7d571 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.207893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1059/2116] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:31.211891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.227148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea94f8-df97-4a0c-a1e9-8d16aac9b0c1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.232846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.234043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.234332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.234652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] PATH_INFO: `/v3/projects/7ef859014e4d4ff79fb7958511f20d11/users/5ad6490c8b8249b1b2abcb0f359330df/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.255164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef859014e4d4ff79fb7958511f20d11, user_id=5ad6490c8b8249b1b2abcb0f359330df, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.257850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.281961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1058/2117] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:31 2022] PUT /identity/v3/projects/7ef859014e4d4ff79fb7958511f20d11/users/5ad6490c8b8249b1b2abcb0f359330df/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:31.283908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.284214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.284498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.313275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:31.314601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:31.391396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1060/2118] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:31.401468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.403310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:31.470079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b73e0-ce28-4724-a742-6a9fc1fff3ea admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.474296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748a03e-d69a-4723-8349-bfa8ca83e7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.474954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a03e-d69a-4723-8349-bfa8ca83e7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.475234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a03e-d69a-4723-8349-bfa8ca83e7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.475536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a03e-d69a-4723-8349-bfa8ca83e7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.486288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748a03e-d69a-4723-8349-bfa8ca83e7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.487791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748a03e-d69a-4723-8349-bfa8ca83e7c4 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.498570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1059/2119] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:31.502076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.504146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:31.519698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278fb3-5e25-4d1e-9f4f-0938cbb7a7ea None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.528760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.530289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.530769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.531003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] PATH_INFO: `/v3/projects/e25f8b8d3790465a8f626d2218d89c42` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.534469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e25f8b8d3790465a8f626d2218d89c42)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.534852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.588138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e25f8b8d3790465a8f626d2218d89c42', 'request_id': 'req-0486a161-5453-4dd6-b416-7d07ef37ab43'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:31.596644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e25f8b8d3790465a8f626d2218d89c42', 'request_id': 'req-0486a161-5453-4dd6-b416-7d07ef37ab43'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:31.603292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e25f8b8d3790465a8f626d2218d89c42', 'request_id': 'req-0486a161-5453-4dd6-b416-7d07ef37ab43'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:31.617651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] The token cache is being invalidate because project e25f8b8d3790465a8f626d2218d89c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:31.617948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:31.618890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0486a161-5453-4dd6-b416-7d07ef37ab43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:31.620232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1061/2120] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:31 2022] DELETE /identity/v3/projects/e25f8b8d3790465a8f626d2218d89c42 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:31.625934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.626226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.626504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.645374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:31.646591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:31.710444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1060/2121] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:31.717911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.718479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.718835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.737710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:31.738938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:31.747854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1062/2122] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:31.753064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.764708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcd07a-1fe4-4834-8a5c-1bb9bb63636d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.769338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.770005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.770294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.770570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.771332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:31.771639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.773585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.791344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1061/2123] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:31.797039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.813373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd14559-7e4c-4676-98ed-5bb5228411a9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.819236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.820210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.820503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.820784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.821480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.823569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.864893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1063/2124] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:31 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:31.866082 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.879017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f7e24-46d2-4133-b88f-38fbed104ee2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.951275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:31.952142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:31.952547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:31.952826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:31.953536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:31.955504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:31.975152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1062/2125] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:31 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:31.979201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:31.993048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8097467-59d1-4185-8b89-3598cd4193f8 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:31.998085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.000082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.000435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.000698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.001443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.003042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.014374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1064/2126] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:32.021309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.038083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c92f2-092a-4b77-94e6-7c45b83774e5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.043539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa110d-d2cd-4ebe-ab79-52ea3518843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.044244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa110d-d2cd-4ebe-ab79-52ea3518843e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.044531 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa110d-d2cd-4ebe-ab79-52ea3518843e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.044805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa110d-d2cd-4ebe-ab79-52ea3518843e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.046050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa110d-d2cd-4ebe-ab79-52ea3518843e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.047643 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa110d-d2cd-4ebe-ab79-52ea3518843e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.064587 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1063/2127] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:32.066202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.080142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a8244-57f1-4650-b874-3689eb3499c9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.084837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.085863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.086152 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.086429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] PATH_INFO: `/v3/projects/1686e1c3a5654980a41a2c30eb7fdf14/users/3f6a65773d2f4b4885e4dbda8a9bfe82/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.109117 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1686e1c3a5654980a41a2c30eb7fdf14, user_id=3f6a65773d2f4b4885e4dbda8a9bfe82, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.113184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.128871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1065/2128] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:32 2022] PUT /identity/v3/projects/1686e1c3a5654980a41a2c30eb7fdf14/users/3f6a65773d2f4b4885e4dbda8a9bfe82/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:32.139519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda6d60-9888-4c89-9681-91f6dbb429bc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.140058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda6d60-9888-4c89-9681-91f6dbb429bc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.140607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda6d60-9888-4c89-9681-91f6dbb429bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.182861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1dda6d60-9888-4c89-9681-91f6dbb429bc None None] MFA Rules not processed for user `3f6a65773d2f4b4885e4dbda8a9bfe82`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:32.184575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1dda6d60-9888-4c89-9681-91f6dbb429bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:32.267372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1064/2129] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:32.273039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.390668 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b75905-1bd0-42b4-b2e4-fd36bd486b24 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.398007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.398874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.399973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.400292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.401024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.402953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.431170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.431524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.431760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.457049 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1066/2130] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:32 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:32.460733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.472586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] MFA Rules not processed for user `5ad6490c8b8249b1b2abcb0f359330df`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:32.473912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:32.550534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10f981-8ab4-493e-9c97-b9e4cd7fb602 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.555446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.556237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1065/2131] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:32.556355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.556661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.557121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.557821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.560915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.567700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.568892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:32.584440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1067/2132] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:32 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:32.588791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.601636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389cb1-dafb-4cc7-bf49-28ed43feb4ed admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.607303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.609075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.609644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.610123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.611058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.612927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.627505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1068/2133] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:32.629517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.639269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917226d7-c408-4ba0-9459-e22c43ee4d05 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.643880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.644777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.645059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.645335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] PATH_INFO: `/v3/projects/587250bc2f0547e28f1aa31ffaafd017/users/0bbeaeaf831b49259e6d4cf74c9d08f6/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.645575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b2e5e-9646-4908-8178-c99a3a254060 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.649161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.649796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.650170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.650448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.662030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] RBAC: Authorizing `identity:create_grant(project_id=587250bc2f0547e28f1aa31ffaafd017, user_id=0bbeaeaf831b49259e6d4cf74c9d08f6, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.662630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.664053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.664931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.674339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1066/2134] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:32.681647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1069/2135] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:32 2022] PUT /identity/v3/projects/587250bc2f0547e28f1aa31ffaafd017/users/0bbeaeaf831b49259e6d4cf74c9d08f6/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:32.686150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.698524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d66f7f-8e4e-4df4-b05e-aef88bf23086 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.756224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741dc97-6bd3-4366-b7ac-a613a627ea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.756927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741dc97-6bd3-4366-b7ac-a613a627ea0b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.757210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741dc97-6bd3-4366-b7ac-a613a627ea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.757493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741dc97-6bd3-4366-b7ac-a613a627ea0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.758251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741dc97-6bd3-4366-b7ac-a613a627ea0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.759832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741dc97-6bd3-4366-b7ac-a613a627ea0b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.769839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1067/2136] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:32.774395 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.784777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f780a1-f603-4dc5-9741-3c2d4b55021d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:32.789728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:32.790619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.790901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.791178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] PATH_INFO: `/v3/projects/587250bc2f0547e28f1aa31ffaafd017/users/0bbeaeaf831b49259e6d4cf74c9d08f6/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.793330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587250bc2f0547e28f1aa31ffaafd017, user_id=0bbeaeaf831b49259e6d4cf74c9d08f6, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:32.797095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:32.810035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1070/2137] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:32 2022] PUT /identity/v3/projects/587250bc2f0547e28f1aa31ffaafd017/users/0bbeaeaf831b49259e6d4cf74c9d08f6/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:32.825236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:32.825571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:32.825802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:32.863184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] MFA Rules not processed for user `0bbeaeaf831b49259e6d4cf74c9d08f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:32.865111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:32.955413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1068/2138] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:32.975011 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:32.977796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:33.054556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2039-0495-4989-9444-cb828e91a8d4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:33.058937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54980f9f-a020-434c-8fc9-fb7ef5dfc05f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:33.059745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54980f9f-a020-434c-8fc9-fb7ef5dfc05f service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:33.060031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54980f9f-a020-434c-8fc9-fb7ef5dfc05f service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:33.060336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54980f9f-a020-434c-8fc9-fb7ef5dfc05f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:33.071590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54980f9f-a020-434c-8fc9-fb7ef5dfc05f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:33.072911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54980f9f-a020-434c-8fc9-fb7ef5dfc05f service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:33.085415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1071/2139] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:33.176368 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:33.185832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3865b-3033-46cb-b350-8c0b41a7faff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:33.190843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4362c9-0ede-4aac-8f60-586e077b5c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:33.191964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4362c9-0ede-4aac-8f60-586e077b5c56 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:33.194131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4362c9-0ede-4aac-8f60-586e077b5c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:33.194428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4362c9-0ede-4aac-8f60-586e077b5c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:33.260761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4362c9-0ede-4aac-8f60-586e077b5c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:33.262733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4362c9-0ede-4aac-8f60-586e077b5c56 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:33.272147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1069/2140] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:34.095587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:34.096123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:34.096243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:34.113806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] MFA Rules not processed for user `5ad6490c8b8249b1b2abcb0f359330df`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:34.115004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:34.168241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1072/2141] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:45.146006 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:45.146502 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:45.146722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:45.179479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:45.180812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:45.247422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1070/2142] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:46:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:45.306774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:45.308780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:45.410816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e392e-87d6-494a-a343-a827696a3765 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:45.418540 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6461ed-cb31-441e-99b1-c9049454eccb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:45.419470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6461ed-cb31-441e-99b1-c9049454eccb service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:45.419902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6461ed-cb31-441e-99b1-c9049454eccb service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:45.420359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6461ed-cb31-441e-99b1-c9049454eccb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:45.435810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6461ed-cb31-441e-99b1-c9049454eccb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:45.440018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6461ed-cb31-441e-99b1-c9049454eccb service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:45.450235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1073/2143] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:46:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:46:47.237309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:47.301519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfa505-b70c-4bbf-b96c-93e0e077d09f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:47.306834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:47.307849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:47.308223 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:47.308567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] PATH_INFO: `/v3/users/3f6a65773d2f4b4885e4dbda8a9bfe82` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:47.309816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f6a65773d2f4b4885e4dbda8a9bfe82)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:47.311807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:47.365959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f6a65773d2f4b4885e4dbda8a9bfe82'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:47.373148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f6a65773d2f4b4885e4dbda8a9bfe82'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:47.381555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f6a65773d2f4b4885e4dbda8a9bfe82'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:47.382278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f6a65773d2f4b4885e4dbda8a9bfe82'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:47.394605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f6a65773d2f4b4885e4dbda8a9bfe82'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:47.400585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1071/2144] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:47 2022] DELETE /identity/v3/users/3f6a65773d2f4b4885e4dbda8a9bfe82 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:46:47.405503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:47.406047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:47.406047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:47.425031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:47.426673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:47.490125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1074/2145] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:47.499607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:47.501065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:47.561163 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0e8e-3d73-4816-b270-ee1bb0186f04 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:47.566072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:47.566776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:47.567059 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:47.567384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:47.580024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:47.587753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:47.600595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1072/2146] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:48.155242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:48.156724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:48.177766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9243e1-d71a-46d7-82b1-5a695e0d6700 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:48.190511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:48.191704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:48.192119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:48.192521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] PATH_INFO: `/v3/projects/1686e1c3a5654980a41a2c30eb7fdf14` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:48.201630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1686e1c3a5654980a41a2c30eb7fdf14)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:48.206517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:48.281080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1686e1c3a5654980a41a2c30eb7fdf14', 'request_id': 'req-44c8852b-2b61-498c-b09b-3f93beb137e7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.297709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1686e1c3a5654980a41a2c30eb7fdf14', 'request_id': 'req-44c8852b-2b61-498c-b09b-3f93beb137e7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.305961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1686e1c3a5654980a41a2c30eb7fdf14', 'request_id': 'req-44c8852b-2b61-498c-b09b-3f93beb137e7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.317600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] The token cache is being invalidate because project 1686e1c3a5654980a41a2c30eb7fdf14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:48.317924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.318585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.320205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1075/2147] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:48 2022] DELETE /identity/v3/projects/1686e1c3a5654980a41a2c30eb7fdf14 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:48.324115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:48.334854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25eba49-23b8-4f02-92ac-58b73154e8f2 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:48.411919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:48.416567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:48.416930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:48.417287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] PATH_INFO: `/v3/users/0bbeaeaf831b49259e6d4cf74c9d08f6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:48.418710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbeaeaf831b49259e6d4cf74c9d08f6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:48.420779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:48.513986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbeaeaf831b49259e6d4cf74c9d08f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.523759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbeaeaf831b49259e6d4cf74c9d08f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.538078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbeaeaf831b49259e6d4cf74c9d08f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.544279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbeaeaf831b49259e6d4cf74c9d08f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.583394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-70b32fdf-a223-4568-9cfb-04e6f49d1c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbeaeaf831b49259e6d4cf74c9d08f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:48.596414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1073/2148] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:48 2022] DELETE /identity/v3/users/0bbeaeaf831b49259e6d4cf74c9d08f6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:48.916150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:48.917663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:48.968032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8852b-2b61-498c-b09b-3f93beb137e7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.050591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.051472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.051948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.052417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] PATH_INFO: `/v3/projects/587250bc2f0547e28f1aa31ffaafd017` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.054305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=587250bc2f0547e28f1aa31ffaafd017)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.056638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.120479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587250bc2f0547e28f1aa31ffaafd017', 'request_id': 'req-820ec8e9-d27f-4753-b8b5-69c60a719b5b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:49.129475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587250bc2f0547e28f1aa31ffaafd017', 'request_id': 'req-820ec8e9-d27f-4753-b8b5-69c60a719b5b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:49.139757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587250bc2f0547e28f1aa31ffaafd017', 'request_id': 'req-820ec8e9-d27f-4753-b8b5-69c60a719b5b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:49.159528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] The token cache is being invalidate because project 587250bc2f0547e28f1aa31ffaafd017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:49.160120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:49.161003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-820ec8e9-d27f-4753-b8b5-69c60a719b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:49.162700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1076/2149] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:48 2022] DELETE /identity/v3/projects/587250bc2f0547e28f1aa31ffaafd017 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:49.169717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.170076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.170414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.191171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:49.192654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:49.279949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1074/2150] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:49.288675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.288962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.289242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.327859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:49.328920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:49.352657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1077/2151] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:49.353177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:49.367542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe6d96-6dbb-4dd6-ac29-3bc41feeb056 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.377904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.379712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.383411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.383811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.384811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:49.385226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.393842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.421857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1075/2152] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:46:49.426714 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:49.440632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25438456-32fe-40f0-8964-635a1ccf5fe8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.448515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.450899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.451537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.453269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.455051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.461278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.520449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1078/2153] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:49 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:49.524867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:49.538932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a322b-4172-40dd-97eb-cbe868e98394 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.702305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.702962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.703245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.703649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.704383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.708035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.741324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1076/2154] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:49 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:49.742257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:49.769489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3bae2-b7fa-43b6-9290-5cbfc3af3bef admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.804980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.805766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.806068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.806476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.807267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.808987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.826858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1079/2155] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:49.832137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:49.848252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0da05a-640c-4c4a-b398-7f80973fdcaf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.863759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11d75b-d763-4cb4-b0ed-1be9f967e7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.864463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11d75b-d763-4cb4-b0ed-1be9f967e7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.864745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11d75b-d763-4cb4-b0ed-1be9f967e7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.865017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11d75b-d763-4cb4-b0ed-1be9f967e7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.865757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11d75b-d763-4cb4-b0ed-1be9f967e7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.868499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11d75b-d763-4cb4-b0ed-1be9f967e7e9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.893061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1077/2156] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:49.897518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:49.913654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7624d-ce47-4ba9-a952-5bc1f21be511 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:49.919720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:49.921244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.921636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.922020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] PATH_INFO: `/v3/projects/249fe09414d646edb2a60131d8b96ad3/users/a0334885ff8c414eb6d0805056224ad4/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:49.945387 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249fe09414d646edb2a60131d8b96ad3, user_id=a0334885ff8c414eb6d0805056224ad4, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:49.950430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:49.975046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1080/2157] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:49 2022] PUT /identity/v3/projects/249fe09414d646edb2a60131d8b96ad3/users/a0334885ff8c414eb6d0805056224ad4/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:49.985600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:49.986196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:49.986637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.023389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] MFA Rules not processed for user `a0334885ff8c414eb6d0805056224ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:50.024838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:50.155834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1078/2158] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:50.175716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:50.358982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea701e78-7d6f-44b3-b271-761b750f3b57 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:50.375223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:50.376535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.377110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.377544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.378262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:50.380252 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:50.426537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1081/2159] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:50 2022] POST /identity/v3/projects => generated 379 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:50.428901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:50.442658 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18f06-c696-4191-b488-16a858b48bde None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:50.514385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:50.515149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.515504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.515853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.516656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:50.520300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:50.551406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1079/2160] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:50 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:50.556465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:50.571140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498419c-fac2-4840-bd96-bf15c9cd2c1d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:50.577854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:50.578729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.579091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.579442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.583987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:50.583987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:50.608964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1082/2161] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:50.617516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:50.634414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d71e3e-c9b2-4d76-9625-4d170a811d6e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:50.644508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec500d8-ef80-4e0a-acc7-29cba548ea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:50.646319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec500d8-ef80-4e0a-acc7-29cba548ea60 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.646908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec500d8-ef80-4e0a-acc7-29cba548ea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.647186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec500d8-ef80-4e0a-acc7-29cba548ea60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.647940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec500d8-ef80-4e0a-acc7-29cba548ea60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:50.649733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec500d8-ef80-4e0a-acc7-29cba548ea60 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:50.668581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1080/2162] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:50.673045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:50.694143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0309-f5be-4007-b5e7-a5ca860fe2b3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:50.709261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:50.710239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.710538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.710942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] PATH_INFO: `/v3/projects/4cdcb6eb6dfd415ebd1dd0fc03a4b6a6/users/fd5eecade6354576bbfa9690e560d7b0/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.744817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdcb6eb6dfd415ebd1dd0fc03a4b6a6, user_id=fd5eecade6354576bbfa9690e560d7b0, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:50.747491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:50.763867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1083/2163] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:50 2022] PUT /identity/v3/projects/4cdcb6eb6dfd415ebd1dd0fc03a4b6a6/users/fd5eecade6354576bbfa9690e560d7b0/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:50.775325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.775589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.775839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.807650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] MFA Rules not processed for user `fd5eecade6354576bbfa9690e560d7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:50.808981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:50.887012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1081/2164] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:50.891232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:50.980830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608475f-9089-4607-bc25-ecc72f611811 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:50.992210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:50.993139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:50.993434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:50.993710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:50.994441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:50.996432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.031235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1084/2165] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:50 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:51.035346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.046106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ffb7d-2f95-4ad8-bccd-90993c0e2241 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.112496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.113316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.113664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.114060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.114854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.117397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.151324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.151789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1082/2166] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:51 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:51.162570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434794a-03e7-4b1f-bfc6-1f2432899b3b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.177498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.178548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.179213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.179753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.181047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.184565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.197660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1085/2167] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:51.201848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.216397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32327473-9c3c-473f-900e-e3359eabd355 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.226846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.227996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.228100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.229623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] PATH_INFO: `/v3/projects/1e43fe6be6e748a8802fe6544beda4c3/users/912758f4f6e04fbfbe86595f751a4aaf/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.255702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e43fe6be6e748a8802fe6544beda4c3, user_id=912758f4f6e04fbfbe86595f751a4aaf, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.258958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.278462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1083/2168] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:51 2022] PUT /identity/v3/projects/1e43fe6be6e748a8802fe6544beda4c3/users/912758f4f6e04fbfbe86595f751a4aaf/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:51.284879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.310027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bf8ac-8eca-4a1a-99d7-fa8f9f8859aa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.397582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b39f6-c276-4957-a95b-47c90cef8f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.400042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b39f6-c276-4957-a95b-47c90cef8f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.400370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b39f6-c276-4957-a95b-47c90cef8f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.400658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b39f6-c276-4957-a95b-47c90cef8f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.402541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b39f6-c276-4957-a95b-47c90cef8f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.405423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b39f6-c276-4957-a95b-47c90cef8f14 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.425340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1086/2169] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:51.429367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.449476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6df25-85a2-4e8b-81da-938ec595a03c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.464307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.464869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.465148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.465416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] PATH_INFO: `/v3/projects/1e43fe6be6e748a8802fe6544beda4c3/users/912758f4f6e04fbfbe86595f751a4aaf/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.469398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e43fe6be6e748a8802fe6544beda4c3, user_id=912758f4f6e04fbfbe86595f751a4aaf, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.472410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.489269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1084/2170] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:51 2022] PUT /identity/v3/projects/1e43fe6be6e748a8802fe6544beda4c3/users/912758f4f6e04fbfbe86595f751a4aaf/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:51.500111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.501087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.502623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.539297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] MFA Rules not processed for user `912758f4f6e04fbfbe86595f751a4aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:51.541097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:51.693442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1087/2171] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:51.704367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.708518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:51.785711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719f5a4-26ce-4cb3-b562-9a44a55e5f3d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.789536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.790197 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.790656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.790998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.801198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.802645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.812401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1085/2172] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:51.858465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:51.860043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:51.874756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68eb28-6381-4ecf-a96a-e407a6d118c2 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:51.880013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e35c8-bd79-44da-84a8-a54ff5e24817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:51.880732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e35c8-bd79-44da-84a8-a54ff5e24817 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:51.881245 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e35c8-bd79-44da-84a8-a54ff5e24817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:51.881656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e35c8-bd79-44da-84a8-a54ff5e24817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:51.944311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e35c8-bd79-44da-84a8-a54ff5e24817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:51.945806 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e35c8-bd79-44da-84a8-a54ff5e24817 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:51.955737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1088/2173] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:54.991601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:54.993268 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:55.086271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713582c5-b1d5-4480-85a2-3afed9981de3 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:55.090853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:55.091711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:55.092001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:55.092278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] PATH_INFO: `/v3/users/c2eceeaba87d4fdab4963f35f5cd7552` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:55.093597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2eceeaba87d4fdab4963f35f5cd7552)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:55.096023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:55.182729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2eceeaba87d4fdab4963f35f5cd7552'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.197850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2eceeaba87d4fdab4963f35f5cd7552'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.214381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2eceeaba87d4fdab4963f35f5cd7552'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.215103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2eceeaba87d4fdab4963f35f5cd7552'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.237463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2eceeaba87d4fdab4963f35f5cd7552'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.261557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1086/2174] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:54 2022] DELETE /identity/v3/users/c2eceeaba87d4fdab4963f35f5cd7552 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:55.265256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:55.265793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:55.265793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:55.297867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:55.299176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:55.363397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:55.365284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:55.416496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1089/2175] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:55.436493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:55.443019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:55.516901 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33245e5-197f-4772-a1b4-92ea059db30a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:55.524894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:55.526231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:55.527210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:55.527780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] PATH_INFO: `/v3/users/a0334885ff8c414eb6d0805056224ad4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:55.529461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0334885ff8c414eb6d0805056224ad4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:55.534885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:55.579476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ec7d6-5733-4f9f-a28d-ec6dbce281c1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:55.597502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9643f6-8c67-432e-a92f-ef4858cd5469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:55.599345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9643f6-8c67-432e-a92f-ef4858cd5469 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:55.599895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9643f6-8c67-432e-a92f-ef4858cd5469 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:55.600544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9643f6-8c67-432e-a92f-ef4858cd5469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:55.634435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9643f6-8c67-432e-a92f-ef4858cd5469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:55.649568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9643f6-8c67-432e-a92f-ef4858cd5469 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:55.671065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0334885ff8c414eb6d0805056224ad4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.679458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0334885ff8c414eb6d0805056224ad4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.682256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1090/2176] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:55.696041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0334885ff8c414eb6d0805056224ad4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.696785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0334885ff8c414eb6d0805056224ad4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.754203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c96bd121-6809-49f4-b8fd-9914241e11b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0334885ff8c414eb6d0805056224ad4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:55.774296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1087/2177] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:55 2022] DELETE /identity/v3/users/a0334885ff8c414eb6d0805056224ad4 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:55.774581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433308-eb60-410e-b243-35b7372088d3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:55.774954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433308-eb60-410e-b243-35b7372088d3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:55.775238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433308-eb60-410e-b243-35b7372088d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:55.809450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-18433308-eb60-410e-b243-35b7372088d3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:55.810812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-18433308-eb60-410e-b243-35b7372088d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:55.902631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:55.903486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:55.905390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:55.942895 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:55.944458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:55.946939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1091/2178] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:55.966781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18433308-eb60-410e-b243-35b7372088d3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:55.972723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-18433308-eb60-410e-b243-35b7372088d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.042739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1088/2179] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:46:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:56.068021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:56.072971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.075381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18433308-eb60-410e-b243-35b7372088d3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.087985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:56.089305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.089673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.090136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.103819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:56.105893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:56.120353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1092/2180] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:56.165083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e749320-de5c-4e45-9cc5-e7da1305f27a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.172146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:56.172772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.173157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.173758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.185260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:56.187090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:56.197871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1089/2181] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:46:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:46:56.282608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:56.285427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.300679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee21bd7-208e-4cf3-b045-7094513c6f88 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.331106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:56.333908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.341548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.343136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] PATH_INFO: `/v3/projects/9564780280f0425883d6d4b2eabb5991` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.351156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9564780280f0425883d6d4b2eabb5991)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:56.354634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:56.552563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9564780280f0425883d6d4b2eabb5991', 'request_id': 'req-75477ad3-e49d-47b9-89e1-4740113a76c5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.557327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:56.558989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.572898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9564780280f0425883d6d4b2eabb5991', 'request_id': 'req-75477ad3-e49d-47b9-89e1-4740113a76c5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.577203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad448fdf-1494-4041-8cc4-3548e9d529ed service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.582339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9564780280f0425883d6d4b2eabb5991', 'request_id': 'req-75477ad3-e49d-47b9-89e1-4740113a76c5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.606596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] The token cache is being invalidate because project 9564780280f0425883d6d4b2eabb5991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:56.607051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.608549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-75477ad3-e49d-47b9-89e1-4740113a76c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.611858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1093/2182] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:56 2022] DELETE /identity/v3/projects/9564780280f0425883d6d4b2eabb5991 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:56.618748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a4df-9501-4529-b114-508dc1a71ec9 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.618943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a4df-9501-4529-b114-508dc1a71ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.619221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a4df-9501-4529-b114-508dc1a71ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.650062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:56.651174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.651536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.651880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] PATH_INFO: `/v3/projects/249fe09414d646edb2a60131d8b96ad3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.653339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249fe09414d646edb2a60131d8b96ad3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:56.655338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:56.659992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f311a4df-9501-4529-b114-508dc1a71ec9 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:56.662922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f311a4df-9501-4529-b114-508dc1a71ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.687904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1094/2183] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:56.698038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.698609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.698884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.741679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:56.743986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.754387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249fe09414d646edb2a60131d8b96ad3', 'request_id': 'req-5813bc6d-4861-4d13-8d06-5952173a3d25'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.770592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249fe09414d646edb2a60131d8b96ad3', 'request_id': 'req-5813bc6d-4861-4d13-8d06-5952173a3d25'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.786680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249fe09414d646edb2a60131d8b96ad3', 'request_id': 'req-5813bc6d-4861-4d13-8d06-5952173a3d25'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.809381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] The token cache is being invalidate because project 249fe09414d646edb2a60131d8b96ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:56.809801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.811671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:56.815882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1090/2184] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:56 2022] DELETE /identity/v3/projects/249fe09414d646edb2a60131d8b96ad3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:56.819956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:56.822747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:56.896555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1095/2185] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:56.897505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:56.919656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01007d39-e523-48d3-92f6-c484aa333b05 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.934356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:56.934790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.935074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.935346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.936094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:56.936423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:56.938042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:56.971527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1096/2186] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:56.972452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:56.976812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813bc6d-4861-4d13-8d06-5952173a3d25 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.989702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988972d4-ecb7-43a3-92bb-90a5bc6a3ab0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:56.993094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:56.993986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:56.994266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:56.995512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] PATH_INFO: `/v3/users/fd5eecade6354576bbfa9690e560d7b0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:56.996937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5eecade6354576bbfa9690e560d7b0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:56.998752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.008009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:57.009846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.010131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.010409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.011107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:57.017467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.079637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1097/2187] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:56 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:57.081184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:57.096849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5eecade6354576bbfa9690e560d7b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:57.106192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5eecade6354576bbfa9690e560d7b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:57.124922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5eecade6354576bbfa9690e560d7b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:57.126194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5eecade6354576bbfa9690e560d7b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:57.152102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5eecade6354576bbfa9690e560d7b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:57.184511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1091/2188] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:56 2022] DELETE /identity/v3/users/fd5eecade6354576bbfa9690e560d7b0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:57.195856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebd72c-cb3e-46af-8d76-1e2d082e562e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:57.314334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:57.315055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.315273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.315550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.316351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:57.318294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.347013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1098/2189] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:57 2022] POST /identity/v3/users => generated 286 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:57.348586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:57.350084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:57.365520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0f570-451f-48c9-8a0b-38fe4cc6962e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:57.374758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:57.375438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.375718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.375989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.377149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:57.379248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.396425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1092/2190] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:57.398519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:57.413206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc575e48-2d97-4474-ae90-bc6e14d2a55f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:57.428282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57617af0-c6d8-4aa1-9a60-b4563e396171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:57.429365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617af0-c6d8-4aa1-9a60-b4563e396171 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.429776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617af0-c6d8-4aa1-9a60-b4563e396171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.430159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617af0-c6d8-4aa1-9a60-b4563e396171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.431033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57617af0-c6d8-4aa1-9a60-b4563e396171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:57.434665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57617af0-c6d8-4aa1-9a60-b4563e396171 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.446705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1099/2191] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:46:57.450808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:57.464544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702c3bd-9e0f-462a-8738-361e27a1562c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:57.475989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984983c-0a4e-454b-93c7-93bdf603b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:57.477055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984983c-0a4e-454b-93c7-93bdf603b6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.477411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984983c-0a4e-454b-93c7-93bdf603b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.478301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984983c-0a4e-454b-93c7-93bdf603b6fc admin admin] PATH_INFO: `/v3/projects/370cf4f142ec4de68eab9ceb0a8ac574/users/b7af784b522c42aba5a2680803564ea2/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.504064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984983c-0a4e-454b-93c7-93bdf603b6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=370cf4f142ec4de68eab9ceb0a8ac574, user_id=b7af784b522c42aba5a2680803564ea2, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:57.507416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984983c-0a4e-454b-93c7-93bdf603b6fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.526322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1093/2192] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:46:57 2022] PUT /identity/v3/projects/370cf4f142ec4de68eab9ceb0a8ac574/users/b7af784b522c42aba5a2680803564ea2/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:57.537253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.537662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.538250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.589199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] MFA Rules not processed for user `b7af784b522c42aba5a2680803564ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:57.590500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:57.780763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.781428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.781428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.782899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1100/2193] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:57.796143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:57.798265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:57.812105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:57.813458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:57.901029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1094/2194] 10.222.0.73 () {58 vars in 967 bytes} [Thu May 12 17:46:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:57.901905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:57.903999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:57.939957 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28a5c7-a18c-40e6-8b5f-3082328d0355 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:57.946659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:57.952015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:57.952355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:57.953785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:57.981293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:57.982741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:57.999108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1101/2195] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:46:58.000244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:58.017151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:58.035170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4611-c502-434d-828b-50fbff96fb3c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:58.040498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:58.041548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.042189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.042713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] PATH_INFO: `/v3/projects/4cdcb6eb6dfd415ebd1dd0fc03a4b6a6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:58.044908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdcb6eb6dfd415ebd1dd0fc03a4b6a6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:58.059426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:58.150282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55204e30-f709-400c-981f-4f943328b2fc service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:58.157946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:58.158846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.159362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.159896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:58.173620 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdcb6eb6dfd415ebd1dd0fc03a4b6a6', 'request_id': 'req-534cf330-4740-415b-bb92-af6468b1d557'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.176681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:58.180297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:58.181052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdcb6eb6dfd415ebd1dd0fc03a4b6a6', 'request_id': 'req-534cf330-4740-415b-bb92-af6468b1d557'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.189558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdcb6eb6dfd415ebd1dd0fc03a4b6a6', 'request_id': 'req-534cf330-4740-415b-bb92-af6468b1d557'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.195264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1102/2196] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:46:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:46:58.197966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:58.219102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] The token cache is being invalidate because project 4cdcb6eb6dfd415ebd1dd0fc03a4b6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:58.219548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.220320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.225314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1095/2197] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:57 2022] DELETE /identity/v3/projects/4cdcb6eb6dfd415ebd1dd0fc03a4b6a6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:58.226114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125dc9a-d7a0-4172-9971-77815a6f740e service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:58.226241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:58.228397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:58.349213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cf330-4740-415b-bb92-af6468b1d557 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:58.371950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:58.373129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.373522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.374091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] PATH_INFO: `/v3/users/b7af784b522c42aba5a2680803564ea2` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:58.375691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:58.376356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7af784b522c42aba5a2680803564ea2)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:58.376720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.377084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.377234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] PATH_INFO: `/v3/users/912758f4f6e04fbfbe86595f751a4aaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:58.378495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] RBAC: Authorizing `identity:delete_user(user_id=912758f4f6e04fbfbe86595f751a4aaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:58.379749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:58.381086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:58.458226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '912758f4f6e04fbfbe86595f751a4aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.475005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '912758f4f6e04fbfbe86595f751a4aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.483598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7af784b522c42aba5a2680803564ea2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.487024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '912758f4f6e04fbfbe86595f751a4aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.487882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '912758f4f6e04fbfbe86595f751a4aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.513028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e2214f-2e82-4cd8-b0c9-b365c74c76af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '912758f4f6e04fbfbe86595f751a4aaf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.522391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7af784b522c42aba5a2680803564ea2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.523482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7af784b522c42aba5a2680803564ea2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.525250 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7af784b522c42aba5a2680803564ea2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.527881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1096/2198] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:58 2022] DELETE /identity/v3/users/912758f4f6e04fbfbe86595f751a4aaf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:58.545490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-954031c7-629e-46f5-a38d-41951a5e051e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7af784b522c42aba5a2680803564ea2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:58.579485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1103/2199] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:46:58 2022] DELETE /identity/v3/users/b7af784b522c42aba5a2680803564ea2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:58.581919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.582208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.582475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:58.643700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:58.647175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:58.828663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.829231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.830441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:58.835280 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1097/2200] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:58.855258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:58.858862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:58.872810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:58.878545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:58.988854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd1631-eb10-474b-98fa-bcb6e744fa5c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:58.993254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:58.993915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:58.994883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:58.995162 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.013650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.015395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.021146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1104/2201] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:46:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:59.026784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.030257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.041897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1098/2202] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:46:59.107556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.107556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.126217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831c34b-5cfc-4bad-b841-5a79d0d37a8e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.138266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.139480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.139748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.140022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] PATH_INFO: `/v3/projects/1e43fe6be6e748a8802fe6544beda4c3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.141736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e43fe6be6e748a8802fe6544beda4c3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.144384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.153350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38a225-5eb3-43ae-b7b5-c5aaed0ece06 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.161920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7c0ea-918e-493e-b24c-6be9a35c4c8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.162820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7c0ea-918e-493e-b24c-6be9a35c4c8d service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.163193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7c0ea-918e-493e-b24c-6be9a35c4c8d service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.163566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7c0ea-918e-493e-b24c-6be9a35c4c8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.179621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7c0ea-918e-493e-b24c-6be9a35c4c8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.181661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7c0ea-918e-493e-b24c-6be9a35c4c8d service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.198321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1105/2203] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:46:59.213508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e43fe6be6e748a8802fe6544beda4c3', 'request_id': 'req-a4f73898-c8bf-41e3-915e-8edaa0accbc8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.224572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e43fe6be6e748a8802fe6544beda4c3', 'request_id': 'req-a4f73898-c8bf-41e3-915e-8edaa0accbc8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.233564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e43fe6be6e748a8802fe6544beda4c3', 'request_id': 'req-a4f73898-c8bf-41e3-915e-8edaa0accbc8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.270426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] The token cache is being invalidate because project 1e43fe6be6e748a8802fe6544beda4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:59.272074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.273127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4f73898-c8bf-41e3-915e-8edaa0accbc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.276467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1099/2204] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:59 2022] DELETE /identity/v3/projects/1e43fe6be6e748a8802fe6544beda4c3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:59.284027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.284383 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.284698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.305179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:59.306906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.386023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1106/2205] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:59.397068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.397763 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.398052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.424138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:59.426105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.452247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1100/2206] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:59.456326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.471196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.472891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.499187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce601d-953a-4d00-9345-6749834aaa2c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.502107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b80228-8a2b-4039-83c5-534403b98beb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.513711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.515279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.515772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.520862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.521220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] PATH_INFO: `/v3/projects/370cf4f142ec4de68eab9ceb0a8ac574` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.521607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.521913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.522190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.528511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=370cf4f142ec4de68eab9ceb0a8ac574)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.530981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:59.531298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.535606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.537901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.559141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1107/2207] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:46:59.567817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.582109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1daec-e830-4ed3-835d-53a8e1adda09 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.597657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.598448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.598831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.599213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.600018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.603206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.619670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370cf4f142ec4de68eab9ceb0a8ac574', 'request_id': 'req-4e323c30-a971-4da6-86c9-3fc6f24599dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.627926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370cf4f142ec4de68eab9ceb0a8ac574', 'request_id': 'req-4e323c30-a971-4da6-86c9-3fc6f24599dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.634067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370cf4f142ec4de68eab9ceb0a8ac574', 'request_id': 'req-4e323c30-a971-4da6-86c9-3fc6f24599dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.659530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] The token cache is being invalidate because project 370cf4f142ec4de68eab9ceb0a8ac574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:46:59.661885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.664455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4e323c30-a971-4da6-86c9-3fc6f24599dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:46:59.665761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1108/2208] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:46:59.669335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1101/2209] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:46:59 2022] DELETE /identity/v3/projects/370cf4f142ec4de68eab9ceb0a8ac574 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:46:59.672092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.676232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175672-70f0-4187-b723-4d840c063af0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.676514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175672-70f0-4187-b723-4d840c063af0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.676788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175672-70f0-4187-b723-4d840c063af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.677528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.726289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-91175672-70f0-4187-b723-4d840c063af0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:59.727554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-91175672-70f0-4187-b723-4d840c063af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.799272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adc8e7-dbfe-4de3-805a-1c8e2c020628 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.805694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.806654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.806939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.807212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.807959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.810633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.852398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1102/2210] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:46:59.862705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.862959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.863242 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.887036 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1109/2211] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:46:59 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:46:59.892391 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.941394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:46:59.942748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:46:59.965122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45852551-19ce-4f17-bea4-1795382a51e4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.968292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1103/2212] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:46:59.972968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.974129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197df37-763e-4c33-955a-597c204489db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.974845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197df37-763e-4c33-955a-597c204489db admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.975273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197df37-763e-4c33-955a-597c204489db admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.975425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197df37-763e-4c33-955a-597c204489db admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.976157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197df37-763e-4c33-955a-597c204489db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.977820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197df37-763e-4c33-955a-597c204489db admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:46:59.982618 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e5984-6e4d-44be-957a-17af722176a4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:46:59.988286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:46:59.988836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:46:59.989053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1110/2213] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:46:59.992013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:46:59.992526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:46:59.993379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:46:59.993505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197df37-763e-4c33-955a-597c204489db admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:46:59.993804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:46:59.995591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.007213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197df37-763e-4c33-955a-597c204489db admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.011998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.013044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.013195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.013392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.013753 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1104/2214] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:47:00.014146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.014789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.015802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.027758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1111/2215] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:47:00.032329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.043705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a667f-b0f7-4170-ad45-30d7bf0a1730 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.049954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.050210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea45de-a9ae-4e19-8eda-4ff0eb985e5f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.051539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.051539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.051539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] PATH_INFO: `/v3/projects/edf90526c5544013ae6d675a9f4fabe5/users/808816bd402547ac9d1d8cf6ce65011b/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.059511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7629648-8eb8-461e-af6f-c4178f051045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.060779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7629648-8eb8-461e-af6f-c4178f051045 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.061131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7629648-8eb8-461e-af6f-c4178f051045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.061479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7629648-8eb8-461e-af6f-c4178f051045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.063314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7629648-8eb8-461e-af6f-c4178f051045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.066131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7629648-8eb8-461e-af6f-c4178f051045 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.067208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf90526c5544013ae6d675a9f4fabe5, user_id=808816bd402547ac9d1d8cf6ce65011b, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.070016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.093207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1112/2216] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:00 2022] PUT /identity/v3/projects/edf90526c5544013ae6d675a9f4fabe5/users/808816bd402547ac9d1d8cf6ce65011b/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:47:00.097582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1105/2217] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:00 2022] POST /identity/v3/projects => generated 391 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:00.107579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.108942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.109404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.109991 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.143043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] MFA Rules not processed for user `808816bd402547ac9d1d8cf6ce65011b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:00.144510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:00.255945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1106/2218] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:00.257101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.308458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27e5e-a074-4315-964d-ec228289ef8b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.350912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281eadf-b3c6-4037-ae34-7e7a06e6e0a5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.355875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.356577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.356854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.357127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.357814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.387544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.402120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.403270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.403667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.404214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.405042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.409735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1107/2219] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:00 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:00.409934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.411074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.443027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1113/2220] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:00 2022] POST /identity/v3/users => generated 296 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:00.444105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.460408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3bebe-c83b-41a8-b2dc-43026dcff8ca admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.508205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677ec1d-895f-41dc-aecf-e2f79b5b8fa3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.520129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.526181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.526686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.526950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.528630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.541049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.576238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1108/2221] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:00 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:00.576438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.576438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.576438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.576438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.576763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.577733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.583952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.594584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc51f80-fbda-420e-a7d0-e53d8c48ab30 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.596991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1114/2222] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:00.603917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.603917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.603917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.606291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.606291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.609132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.625591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1109/2223] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:00.625780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.630680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.641666 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831f0b1-ec83-462c-a734-468b3536f253 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.651478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.651868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53d261-e4ba-4e1f-b55d-29e86f610e6a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.652842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.653341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.653799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] PATH_INFO: `/v3/projects/198347c20f3548f9b0a010beab0c319a/users/5338d68c44f447578937ef25745398f9/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.656598 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.657238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.659013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.659013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.659407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.661052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.671393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1115/2224] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:00.672822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198347c20f3548f9b0a010beab0c319a, user_id=5338d68c44f447578937ef25745398f9, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.675823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.677152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.686982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568ef9-ee62-496b-b86d-57f9ee784875 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.695957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1110/2225] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:00 2022] PUT /identity/v3/projects/198347c20f3548f9b0a010beab0c319a/users/5338d68c44f447578937ef25745398f9/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:00.699774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.755615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7966b-8864-4c26-b66d-c52308803bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.756829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7966b-8864-4c26-b66d-c52308803bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.757361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7966b-8864-4c26-b66d-c52308803bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.757879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7966b-8864-4c26-b66d-c52308803bd4 admin admin] PATH_INFO: `/v3/projects/2e2c86afb03e4ad8afd66ef5ff9a7ec7/users/d2cdf17fb3be47cda297fc2c805cc573/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.778267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7966b-8864-4c26-b66d-c52308803bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2c86afb03e4ad8afd66ef5ff9a7ec7, user_id=d2cdf17fb3be47cda297fc2c805cc573, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.782080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7966b-8864-4c26-b66d-c52308803bd4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.797577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798a9f6-7df3-4244-9308-791e086bd406 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.801784 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1116/2226] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:00 2022] PUT /identity/v3/projects/2e2c86afb03e4ad8afd66ef5ff9a7ec7/users/d2cdf17fb3be47cda297fc2c805cc573/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:47:00.812639 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.813185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.813523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.842611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] MFA Rules not processed for user `d2cdf17fb3be47cda297fc2c805cc573`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:00.843964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:00.921886 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.922610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.922906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.923733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.924571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.926530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.950144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1111/2227] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:47:00.951821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.967730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad03cf-0743-4cea-8f3c-6ee6b15580a9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:00.973883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1117/2228] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:47:00.975538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cd1ad-e891-4547-af71-d1e35d4bc8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:00.976525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cd1ad-e891-4547-af71-d1e35d4bc8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:00.977052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cd1ad-e891-4547-af71-d1e35d4bc8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:00.977485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cd1ad-e891-4547-af71-d1e35d4bc8e4 admin admin] PATH_INFO: `/v3/projects/198347c20f3548f9b0a010beab0c319a/users/5338d68c44f447578937ef25745398f9/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:00.980444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cd1ad-e891-4547-af71-d1e35d4bc8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198347c20f3548f9b0a010beab0c319a, user_id=5338d68c44f447578937ef25745398f9, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:00.984676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cd1ad-e891-4547-af71-d1e35d4bc8e4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:00.985251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:00.988214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:00.998826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1112/2229] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:00 2022] PUT /identity/v3/projects/198347c20f3548f9b0a010beab0c319a/users/5338d68c44f447578937ef25745398f9/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:01.011907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:01.012244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:01.012701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:01.051532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] MFA Rules not processed for user `5338d68c44f447578937ef25745398f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:01.053001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:01.096604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9601d-ea64-4813-8aa6-7ff04f6a6a9c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:01.172230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1113/2230] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:47:01.190875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:01.191428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:01.191744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:01.191988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:01.251669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:01.253364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:01.263386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1118/2231] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:47:01.552311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:01.565467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f925ea6-efd3-4fd7-a290-7b747415841a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:01.569968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08755bd0-2528-4c1c-a3ce-e07787fdde07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:01.570865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08755bd0-2528-4c1c-a3ce-e07787fdde07 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:01.571320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08755bd0-2528-4c1c-a3ce-e07787fdde07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:01.571775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08755bd0-2528-4c1c-a3ce-e07787fdde07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:01.584874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08755bd0-2528-4c1c-a3ce-e07787fdde07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:01.587285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08755bd0-2528-4c1c-a3ce-e07787fdde07 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:01.602291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1114/2232] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:02.252214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:02.346582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220957aa-1cbd-4a08-a9bf-1a6cf3ef2217 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:02.360193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:02.360193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:02.360193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:02.360193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] PATH_INFO: `/v3/users/808816bd402547ac9d1d8cf6ce65011b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:02.360997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=808816bd402547ac9d1d8cf6ce65011b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:02.360997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:02.406153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '808816bd402547ac9d1d8cf6ce65011b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:02.418725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '808816bd402547ac9d1d8cf6ce65011b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:02.419438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '808816bd402547ac9d1d8cf6ce65011b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:02.420002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '808816bd402547ac9d1d8cf6ce65011b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:02.427880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '808816bd402547ac9d1d8cf6ce65011b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:02.442314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1119/2233] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:02 2022] DELETE /identity/v3/users/808816bd402547ac9d1d8cf6ce65011b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:02.447785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:02.448494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:02.449113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:02.468948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:02.470471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:02.542101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1115/2234] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:02.553411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:02.555302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:02.692779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5244cc0-fa9c-4ea8-aadf-1f2638680f07 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:02.697471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:02.698321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:02.698442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:02.698685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:02.708837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:02.710326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:02.720779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1120/2235] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:03.079463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:03.082289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:03.107698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214346c-9106-4dac-b8ec-5da2acb094ed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:03.114384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:03.115281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:03.115555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:03.115830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] PATH_INFO: `/v3/projects/edf90526c5544013ae6d675a9f4fabe5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:03.117111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf90526c5544013ae6d675a9f4fabe5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:03.118976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:03.183521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf90526c5544013ae6d675a9f4fabe5', 'request_id': 'req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.191175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf90526c5544013ae6d675a9f4fabe5', 'request_id': 'req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.197806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf90526c5544013ae6d675a9f4fabe5', 'request_id': 'req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.218227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] The token cache is being invalidate because project edf90526c5544013ae6d675a9f4fabe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:03.218621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.219360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.221858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1116/2236] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:03 2022] DELETE /identity/v3/projects/edf90526c5544013ae6d675a9f4fabe5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:03.226576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:03.237799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a9fae-3b9e-4728-a7af-2528091c58db service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:03.298292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:03.299333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:03.299504 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:03.299845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] PATH_INFO: `/v3/users/5338d68c44f447578937ef25745398f9` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:03.301135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5338d68c44f447578937ef25745398f9)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:03.303110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:03.352003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5338d68c44f447578937ef25745398f9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.366248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5338d68c44f447578937ef25745398f9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.367236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5338d68c44f447578937ef25745398f9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.368451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5338d68c44f447578937ef25745398f9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.377475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6fe5f8af-d069-4519-b0ad-d3337c21c73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5338d68c44f447578937ef25745398f9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.394314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1121/2237] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:03 2022] DELETE /identity/v3/users/5338d68c44f447578937ef25745398f9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:47:03.813533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:03.815649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:03.841998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a3af-c555-48e2-a4f9-c4116a29a1fc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:03.931376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:03.932423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:03.932821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:03.933103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] PATH_INFO: `/v3/projects/198347c20f3548f9b0a010beab0c319a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:03.934292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=198347c20f3548f9b0a010beab0c319a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:03.936339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:03.985433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198347c20f3548f9b0a010beab0c319a', 'request_id': 'req-b2581bb8-2023-412a-b963-d3d8f73748b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.992062 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198347c20f3548f9b0a010beab0c319a', 'request_id': 'req-b2581bb8-2023-412a-b963-d3d8f73748b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:03.997821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198347c20f3548f9b0a010beab0c319a', 'request_id': 'req-b2581bb8-2023-412a-b963-d3d8f73748b2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:04.011781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] The token cache is being invalidate because project 198347c20f3548f9b0a010beab0c319a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:04.012074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:04.012762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b2581bb8-2023-412a-b963-d3d8f73748b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:04.014610 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1117/2238] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:03 2022] DELETE /identity/v3/projects/198347c20f3548f9b0a010beab0c319a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:04.020863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.021161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.021539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.043572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:04.044924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:04.113996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1122/2239] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:04.122737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.123217 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.123295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.158842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:04.161762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:04.174782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1118/2240] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:04.179234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.192738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d630c00-07bf-475f-be47-478e687d6d13 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.194789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.195447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.195726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.196116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.196881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:47:04.197237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.198807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.213759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1123/2241] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:47:04.218472 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.228673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc419ab-9a7a-411d-b86a-80595e492baa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.234693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.235365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.236214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.236456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.237150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.239095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.268131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1119/2242] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:04.272516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.283537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07674378-c8c9-4369-828b-accf59e15c37 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.348834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.349525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.349774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.350039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.350734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.352669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.373299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1124/2243] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:47:04.377596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.389805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0d229-ed25-4231-b527-448d45028b27 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.395977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.396633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.397009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.397416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.398116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.400017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.415594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1120/2244] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:47:04.416747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.426910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5631-5e7a-479b-9d50-9aaf84c0f87a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.432525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccda298-6a19-4744-bb73-fe6d2c26c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.433300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccda298-6a19-4744-bb73-fe6d2c26c452 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.433801 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccda298-6a19-4744-bb73-fe6d2c26c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.434123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccda298-6a19-4744-bb73-fe6d2c26c452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.435304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccda298-6a19-4744-bb73-fe6d2c26c452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.437448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccda298-6a19-4744-bb73-fe6d2c26c452 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.451157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1125/2245] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:47:04.455259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.465839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910300e8-4717-43d8-85e5-7e32d2b80c72 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.471487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.472396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.472692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.472953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] PATH_INFO: `/v3/projects/80e4b45302a54c37b22648c90bdfdaae/users/bb72ef8c24e74106baf83ae8b7c6214a/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.490987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e4b45302a54c37b22648c90bdfdaae, user_id=bb72ef8c24e74106baf83ae8b7c6214a, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.493801 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.511656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1121/2246] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:04 2022] PUT /identity/v3/projects/80e4b45302a54c37b22648c90bdfdaae/users/bb72ef8c24e74106baf83ae8b7c6214a/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:04.522820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.523157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.523442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.552712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] MFA Rules not processed for user `bb72ef8c24e74106baf83ae8b7c6214a`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:04.554123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:04.650469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1126/2247] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:04.654790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.739970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf0a09-5d61-44ef-8fff-edb2434ea5da admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.746153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.746919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.747261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.747636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.748468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.750856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.784911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1122/2248] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:47:04.786054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.797059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4208f-cb57-4252-8e7c-47b3a5aaf99e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.875496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.876895 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.877390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.877505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.878275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.880270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.904374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1127/2249] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:04 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:04.907226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.922975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd70b-1e37-49e6-b7e6-1d25c5ff0202 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.929073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.929701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.930115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.930276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.931030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.932699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:04.944564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1123/2250] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:04.949163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:04.962791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef0ad2-fbfa-44c5-82a2-e941948c2fdd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:04.968627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:04.970335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:04.970730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:04.971184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] PATH_INFO: `/v3/projects/90b9fdd2117a499fb6f012dd7bd0ed5c/users/5cd1ec28b45046f3b65056faee8089c6/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:04.990724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b9fdd2117a499fb6f012dd7bd0ed5c, user_id=5cd1ec28b45046f3b65056faee8089c6, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:04.993870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:05.019796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1128/2251] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:04 2022] PUT /identity/v3/projects/90b9fdd2117a499fb6f012dd7bd0ed5c/users/5cd1ec28b45046f3b65056faee8089c6/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:05.023759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:05.036495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93aa0-bcc1-4d98-a720-13ca3ca5b770 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:05.132440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2affd2-a38d-48c6-af4c-ca755208940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:05.133263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2affd2-a38d-48c6-af4c-ca755208940e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:05.133670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2affd2-a38d-48c6-af4c-ca755208940e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:05.134113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2affd2-a38d-48c6-af4c-ca755208940e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:05.135105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2affd2-a38d-48c6-af4c-ca755208940e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:05.137091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2affd2-a38d-48c6-af4c-ca755208940e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:05.152079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1124/2252] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:05.153307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:05.164459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3793aba-d12c-4085-98e6-baff82e91cd7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:05.169280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:05.170181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:05.170469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:05.170747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] PATH_INFO: `/v3/projects/90b9fdd2117a499fb6f012dd7bd0ed5c/users/5cd1ec28b45046f3b65056faee8089c6/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:05.173026 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b9fdd2117a499fb6f012dd7bd0ed5c, user_id=5cd1ec28b45046f3b65056faee8089c6, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:05.176098 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:05.195737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1129/2253] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:05 2022] PUT /identity/v3/projects/90b9fdd2117a499fb6f012dd7bd0ed5c/users/5cd1ec28b45046f3b65056faee8089c6/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:05.207316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:05.207679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:05.208037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:05.250020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] MFA Rules not processed for user `5cd1ec28b45046f3b65056faee8089c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:05.251738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:05.374077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1125/2254] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:05.385418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:05.387485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:05.489944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a5b6a-5d1e-4f1f-9e3e-e1ad6ddada51 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:05.503642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:05.508517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:05.508847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:05.509102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:05.526435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:05.528100 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:05.539110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1130/2255] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:05.663450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:05.673982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d822c-bce1-4192-8fd4-f2a4f624a359 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:05.678001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1a183-69e6-4b80-9847-60c7c2ef8eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:05.678689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1a183-69e6-4b80-9847-60c7c2ef8eab service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:05.679080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1a183-69e6-4b80-9847-60c7c2ef8eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:05.679355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1a183-69e6-4b80-9847-60c7c2ef8eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:05.747686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1a183-69e6-4b80-9847-60c7c2ef8eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:05.749157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1a183-69e6-4b80-9847-60c7c2ef8eab service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:05.759300 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1126/2256] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:06.122953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:06.255366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d536-3b35-4a98-a7c2-1dad7df649d0 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:06.263328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:06.264655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:06.265239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:06.265727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] PATH_INFO: `/v3/users/bb72ef8c24e74106baf83ae8b7c6214a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:06.275075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb72ef8c24e74106baf83ae8b7c6214a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:06.277871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:06.356940 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb72ef8c24e74106baf83ae8b7c6214a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:06.371156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb72ef8c24e74106baf83ae8b7c6214a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:06.372747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb72ef8c24e74106baf83ae8b7c6214a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:06.373378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb72ef8c24e74106baf83ae8b7c6214a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:06.380961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb72ef8c24e74106baf83ae8b7c6214a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:06.396031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1131/2257] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:06 2022] DELETE /identity/v3/users/bb72ef8c24e74106baf83ae8b7c6214a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:06.400626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:06.401076 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:06.401183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:06.424441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:06.425664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:06.492542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1127/2258] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:06.499427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:06.501339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:06.578938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86bbc7-d6b5-4f89-91ad-17bf1d319e36 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:06.584881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:06.585850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:06.586304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:06.586788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:06.599265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:06.600894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:06.618263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1132/2259] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:06.902899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:06.906109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:06.925971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d9d3b-6230-43db-a284-789c9ef1bfd5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:06.931003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:06.931870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:06.932147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:06.932451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] PATH_INFO: `/v3/projects/80e4b45302a54c37b22648c90bdfdaae` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:06.933653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e4b45302a54c37b22648c90bdfdaae)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:06.935565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:06.994720 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e4b45302a54c37b22648c90bdfdaae', 'request_id': 'req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.005740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e4b45302a54c37b22648c90bdfdaae', 'request_id': 'req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.017486 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e4b45302a54c37b22648c90bdfdaae', 'request_id': 'req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.036211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] The token cache is being invalidate because project 80e4b45302a54c37b22648c90bdfdaae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:07.036540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.037374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.039041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1128/2260] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:06 2022] DELETE /identity/v3/projects/80e4b45302a54c37b22648c90bdfdaae => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:07.049385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:07.059263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5255-026a-4ea4-bd6a-12e140f736f6 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:07.152393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:07.155870 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:07.158564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:07.158831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] PATH_INFO: `/v3/users/5cd1ec28b45046f3b65056faee8089c6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:07.160260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd1ec28b45046f3b65056faee8089c6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:07.162529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:07.230585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd1ec28b45046f3b65056faee8089c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.250094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd1ec28b45046f3b65056faee8089c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.253851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd1ec28b45046f3b65056faee8089c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.255241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd1ec28b45046f3b65056faee8089c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.277109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d2a8489-5591-429c-8c19-e7f1a2e9ea0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd1ec28b45046f3b65056faee8089c6'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.316022 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1133/2261] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:07 2022] DELETE /identity/v3/users/5cd1ec28b45046f3b65056faee8089c6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:07.769856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:07.771513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:07.803494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a3ca7-221c-4b1e-9acd-c90d7dcc9701 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:07.875414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:07.877208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:07.877330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:07.877557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] PATH_INFO: `/v3/projects/90b9fdd2117a499fb6f012dd7bd0ed5c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:07.878770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b9fdd2117a499fb6f012dd7bd0ed5c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:07.880704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:07.929568 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b9fdd2117a499fb6f012dd7bd0ed5c', 'request_id': 'req-916cd99c-a920-4891-aad9-c610d90415c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.935721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b9fdd2117a499fb6f012dd7bd0ed5c', 'request_id': 'req-916cd99c-a920-4891-aad9-c610d90415c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.943281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b9fdd2117a499fb6f012dd7bd0ed5c', 'request_id': 'req-916cd99c-a920-4891-aad9-c610d90415c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.960526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] The token cache is being invalidate because project 90b9fdd2117a499fb6f012dd7bd0ed5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:07.961259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.962093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-916cd99c-a920-4891-aad9-c610d90415c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:07.963776 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1129/2262] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:07 2022] DELETE /identity/v3/projects/90b9fdd2117a499fb6f012dd7bd0ed5c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:07.969936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:07.970552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:07.970701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:07.992329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:07.993579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:08.073436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1134/2263] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:08.086341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.086341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.086341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.109335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:08.111465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:08.124314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1130/2264] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:08.124809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.135165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3fd00-e710-4b99-bacb-2acb25266353 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.139850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.140536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.140818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.141096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.142000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:47:08.142318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.143928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.156137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1135/2265] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:47:08.160994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.171019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4523587-8ff9-4138-8c52-ea94fab921f4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.177181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.177827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.178129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.178397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.179173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.182256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.212257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1131/2266] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:08 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:08.215958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.227457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292cc1-e236-481e-a034-8663edb31c3b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.287959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.288697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.288991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.289269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.290109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.292041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.322724 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1136/2267] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:08 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:47:08.327052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.337717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6eddb-ebbb-4668-98c3-17905e3c9815 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.342386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.343040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.343318 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.343590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.344348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.346549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.361807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.364471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1132/2268] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:08.372059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda85c3-2730-4912-8d05-b7bc623b55ae admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.378231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcbade-f5ca-4743-9204-576be211eb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.378994 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcbade-f5ca-4743-9204-576be211eb44 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.379535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcbade-f5ca-4743-9204-576be211eb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.379823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcbade-f5ca-4743-9204-576be211eb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.380596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcbade-f5ca-4743-9204-576be211eb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.382570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcbade-f5ca-4743-9204-576be211eb44 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.396515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1137/2269] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:47:08.400579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.412837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a715-9d29-428a-8180-19eb7031f46c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.422252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.423703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.424527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.424527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] PATH_INFO: `/v3/projects/3e0a5b853e6347ffa61e544dddd501a6/users/e0d919cfe3e0434c9833e4013ad217cc/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.442970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0a5b853e6347ffa61e544dddd501a6, user_id=e0d919cfe3e0434c9833e4013ad217cc, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.445722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.472230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1133/2270] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:08 2022] PUT /identity/v3/projects/3e0a5b853e6347ffa61e544dddd501a6/users/e0d919cfe3e0434c9833e4013ad217cc/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:08.474646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f94a0-ddb9-41c3-b78f-53e7f5d30d22 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.475109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f94a0-ddb9-41c3-b78f-53e7f5d30d22 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.475496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f94a0-ddb9-41c3-b78f-53e7f5d30d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.516520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-309f94a0-ddb9-41c3-b78f-53e7f5d30d22 None None] MFA Rules not processed for user `e0d919cfe3e0434c9833e4013ad217cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:08.518808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-309f94a0-ddb9-41c3-b78f-53e7f5d30d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:08.632654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1138/2271] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:08.674083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:08.676313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:08.776462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101322e-abc6-4e1f-951c-b0c1af38b599 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:08.783683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ace3-9884-487e-837a-be8fa04cab62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:08.785136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ace3-9884-487e-837a-be8fa04cab62 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:08.785701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ace3-9884-487e-837a-be8fa04cab62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:08.786455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ace3-9884-487e-837a-be8fa04cab62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:08.800797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ace3-9884-487e-837a-be8fa04cab62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:08.803476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ace3-9884-487e-837a-be8fa04cab62 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:08.818137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1134/2272] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:10.367700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:10.368188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:10.368283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:10.385553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] MFA Rules not processed for user `1cc16878bc6a445f8a66dfcdc54a7e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:10.386747 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:10.441119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1139/2273] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:12.806622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158698-1854-4550-8d46-2a8edb63d221 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:12.807137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158698-1854-4550-8d46-2a8edb63d221 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:12.807214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158698-1854-4550-8d46-2a8edb63d221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:12.825948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-de158698-1854-4550-8d46-2a8edb63d221 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:12.827383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-de158698-1854-4550-8d46-2a8edb63d221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:12.885740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1135/2274] 10.222.0.73 () {58 vars in 967 bytes} [Thu May 12 17:47:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:12.898674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:12.900896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:12.975412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c1fb0-f454-410e-90ed-1fb0eb1c814c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:12.981201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:12.981868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:12.982157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:12.982436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:12.993103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:12.994686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:13.004594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1140/2275] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:47:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:47:17.013012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:17.013482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:17.013593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:17.042032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:17.043253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:17.051189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1136/2276] 10.222.0.73 () {58 vars in 967 bytes} [Thu May 12 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:17.063238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:17.073362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b571d-ed7b-43ec-8975-0500661deafd service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:17.078348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0d763-2446-4ef0-87ed-6b1b24a5fade service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:17.078977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0d763-2446-4ef0-87ed-6b1b24a5fade service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:17.079262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0d763-2446-4ef0-87ed-6b1b24a5fade service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:17.079536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0d763-2446-4ef0-87ed-6b1b24a5fade service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:17.087863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0d763-2446-4ef0-87ed-6b1b24a5fade service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:17.089350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0d763-2446-4ef0-87ed-6b1b24a5fade service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:17.099718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1141/2277] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:47:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:47:17.415164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:17.417010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:17.510191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf37e27-836c-4148-af69-7d7f7710f128 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:17.516393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:17.517529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:17.517861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:17.518174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] PATH_INFO: `/v3/users/2e95c3d471634389a70bfcd7515085ae` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:17.519377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e95c3d471634389a70bfcd7515085ae)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:17.521534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:17.575460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e95c3d471634389a70bfcd7515085ae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:17.585974 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e95c3d471634389a70bfcd7515085ae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:17.593760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e95c3d471634389a70bfcd7515085ae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:17.594373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e95c3d471634389a70bfcd7515085ae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:17.608129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e95c3d471634389a70bfcd7515085ae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:17.617017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1137/2278] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:17 2022] DELETE /identity/v3/users/2e95c3d471634389a70bfcd7515085ae => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:17.621883 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:17.622175 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:17.622454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:17.641881 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:17.643216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:17.735651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1142/2279] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:17.742505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:17.743988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:17.817100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e069b2-98c1-4a17-a204-9f8beac5e8bb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:17.821548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84e8c7-6adc-4f7c-bdd2-b81bc44d4373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:17.824563 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84e8c7-6adc-4f7c-bdd2-b81bc44d4373 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:17.824954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84e8c7-6adc-4f7c-bdd2-b81bc44d4373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:17.825333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84e8c7-6adc-4f7c-bdd2-b81bc44d4373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:17.856192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84e8c7-6adc-4f7c-bdd2-b81bc44d4373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:17.856192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84e8c7-6adc-4f7c-bdd2-b81bc44d4373 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:17.867837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1138/2280] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:18.847371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:18.848867 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:18.886745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a70dad-b11b-4fd4-a090-74c0a1f26b36 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:18.901170 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:18.903183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:18.904404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:18.904760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] PATH_INFO: `/v3/projects/e53ce05ac35c48aeb17842025bb4c455` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:18.906021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53ce05ac35c48aeb17842025bb4c455)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:18.911080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:18.987596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53ce05ac35c48aeb17842025bb4c455', 'request_id': 'req-a277b07f-0809-4e62-ad9a-542e1384da3f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:18.999463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53ce05ac35c48aeb17842025bb4c455', 'request_id': 'req-a277b07f-0809-4e62-ad9a-542e1384da3f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.008313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53ce05ac35c48aeb17842025bb4c455', 'request_id': 'req-a277b07f-0809-4e62-ad9a-542e1384da3f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.020652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] The token cache is being invalidate because project e53ce05ac35c48aeb17842025bb4c455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:19.021012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.021829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a277b07f-0809-4e62-ad9a-542e1384da3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.024061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1143/2281] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:18 2022] DELETE /identity/v3/projects/e53ce05ac35c48aeb17842025bb4c455 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:19.028701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f409a-f7f3-4f23-b931-37ca055e2d89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.029196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f409a-f7f3-4f23-b931-37ca055e2d89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.029293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f409a-f7f3-4f23-b931-37ca055e2d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.061631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-827f409a-f7f3-4f23-b931-37ca055e2d89 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:19.063641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-827f409a-f7f3-4f23-b931-37ca055e2d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:19.141228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1139/2282] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:19.149643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.149932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.150205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.153573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.153993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.154272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.194638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:19.195534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:19.196180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:19.196735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:19.213925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1140/2283] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:19.223088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.233864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6e081-ba9b-4307-ad29-a467308de1c1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.240443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.241208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.242224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.242579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.243384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:47:19.243746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.245929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.270850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1141/2284] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:47:19.271922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.287930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd1b16-8ac5-4fa5-b068-f3fa9f95881f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.289082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1144/2285] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:19.293078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.293806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.294160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.294505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.295384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.297692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.304677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.306052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:19.344288 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1142/2286] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:19 2022] POST /identity/v3/projects => generated 399 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:47:19.348794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.404886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df70361-3a06-438f-8b24-de0ea2d57f9f None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.522468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6ea8d-8646-4b49-8574-1ec30ffcbf09 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.528330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31181-3124-4033-8676-4078581f07ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.528330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31181-3124-4033-8676-4078581f07ab service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.528330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31181-3124-4033-8676-4078581f07ab service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.528330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b31181-3124-4033-8676-4078581f07ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.534302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.536816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.537864 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.538161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.538870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.540834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.590421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1143/2287] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:19 2022] POST /identity/v3/users => generated 300 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:19.595093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.619624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589392f9-22bf-496e-88fd-a253bd8e8db2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.651601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.651601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.651601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.651601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.652738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.652738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.673181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1144/2288] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:19.675196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b31181-3124-4033-8676-4078581f07ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.690925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.693615 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b31181-3124-4033-8676-4078581f07ab service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.715296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0f1a6-acb6-41aa-a631-51ffe1276766 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.717070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1145/2289] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:47:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:47:19.718032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31181-3124-4033-8676-4078581f07ab service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.724241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-50b31181-3124-4033-8676-4078581f07ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:19.728642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.729528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.729968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.730752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.731598 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.734387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.744869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b31181-3124-4033-8676-4078581f07ab service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.754002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.757215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1145/2290] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:19.757768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.758183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.758309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:19.758495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] PATH_INFO: `/v3/users/9ad11a30da01449f82c1b94c7181860f` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.761796 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad11a30da01449f82c1b94c7181860f)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.779606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.800142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc8c3f-c6f3-441e-a380-96fb73c2d660 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:19.814014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d5890-346c-4dc0-82a1-7ea8917509ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:19.815169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5890-346c-4dc0-82a1-7ea8917509ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.815491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5890-346c-4dc0-82a1-7ea8917509ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.816306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5890-346c-4dc0-82a1-7ea8917509ca admin admin] PATH_INFO: `/v3/projects/11d3e88c3ffe43beb17fbfcb2fbf6702/users/3d73c95e62474cb185bae080248af9da/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.861133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d5890-346c-4dc0-82a1-7ea8917509ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d3e88c3ffe43beb17fbfcb2fbf6702, user_id=3d73c95e62474cb185bae080248af9da, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:19.866687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d5890-346c-4dc0-82a1-7ea8917509ca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:19.882382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad11a30da01449f82c1b94c7181860f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.897300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad11a30da01449f82c1b94c7181860f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.898606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad11a30da01449f82c1b94c7181860f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.899521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad11a30da01449f82c1b94c7181860f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.907612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad11a30da01449f82c1b94c7181860f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:19.911215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1146/2291] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:19 2022] PUT /identity/v3/projects/11d3e88c3ffe43beb17fbfcb2fbf6702/users/3d73c95e62474cb185bae080248af9da/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:19.920249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:19.920602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:19.922180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:19.942812 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1146/2292] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:19 2022] DELETE /identity/v3/users/9ad11a30da01449f82c1b94c7181860f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:20.015622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] MFA Rules not processed for user `3d73c95e62474cb185bae080248af9da`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:20.020668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:20.215296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1147/2293] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:20.353716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:20.355214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:20.526230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ece0-b0f5-4583-b536-ae18a493ee9b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:20.541935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a48e7b-4bdb-414a-b52a-2a05a5a22487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:20.542634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48e7b-4bdb-414a-b52a-2a05a5a22487 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:20.542921 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48e7b-4bdb-414a-b52a-2a05a5a22487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:20.543197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48e7b-4bdb-414a-b52a-2a05a5a22487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:20.563501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a48e7b-4bdb-414a-b52a-2a05a5a22487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:20.566659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a48e7b-4bdb-414a-b52a-2a05a5a22487 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:20.596809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1147/2294] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:20.849462 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:20.851292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:21.066097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9f24-7e1e-4986-9e38-a15cc6a4005e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:21.080073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:21.082616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.082899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.083766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] PATH_INFO: `/v3/projects/887261f594284422bcd448d443d27013` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.087839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] RBAC: Authorizing `identity:delete_project(project_id=887261f594284422bcd448d443d27013)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:21.093440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:21.213131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887261f594284422bcd448d443d27013', 'request_id': 'req-d402e369-a91a-4cb4-8cce-0edf2cc584db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:21.221876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887261f594284422bcd448d443d27013', 'request_id': 'req-d402e369-a91a-4cb4-8cce-0edf2cc584db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:21.243094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887261f594284422bcd448d443d27013', 'request_id': 'req-d402e369-a91a-4cb4-8cce-0edf2cc584db'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:21.263263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] The token cache is being invalidate because project 887261f594284422bcd448d443d27013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:21.263860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:21.264867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d402e369-a91a-4cb4-8cce-0edf2cc584db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:21.267211 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1148/2295] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:20 2022] DELETE /identity/v3/projects/887261f594284422bcd448d443d27013 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:21.273236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.274256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.274691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.305031 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:21.306441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:21.416120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1148/2296] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:21.423381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.424251 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.424613 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.469278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:21.470567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:21.480824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1149/2297] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:21.484669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:21.500195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b63976-1a04-4796-ba95-fd6388c72d35 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:21.522242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:21.523223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.523787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.524427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.525718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:47:21.526347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:21.528978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:21.553551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1149/2298] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:47:21.554743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:21.567637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4b9f3-73a5-4bc9-807e-713a669ffded None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:21.578867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:21.579537 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.579813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.580117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.580833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:21.582736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:21.674294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1150/2299] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:47:21 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:21.678721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:21.703771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d6c02-bc94-451d-bf2a-6e3a5bef5a51 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:21.850606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:21.851392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.851751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.852101 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.852954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:21.855608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:21.908960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1150/2300] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:47:21 2022] POST /identity/v3/users => generated 284 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:47:21.912224 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:21.929848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b52db8-8b75-4877-b8c3-2e2fe7fa7c0c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:21.945755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:21.946553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:21.946926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:21.947860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:21.948785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:21.950485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:21.975241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1151/2301] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:21.983549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:22.000764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63587b9c-7714-4f8e-80fa-ef1b621b8d52 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:22.020595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a1cb0-27cb-4470-a56e-199baa31d157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:22.021322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a1cb0-27cb-4470-a56e-199baa31d157 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:22.021584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a1cb0-27cb-4470-a56e-199baa31d157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:22.021905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a1cb0-27cb-4470-a56e-199baa31d157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:22.022690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a1cb0-27cb-4470-a56e-199baa31d157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:22.024968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a1cb0-27cb-4470-a56e-199baa31d157 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:22.061611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1151/2302] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:47:22.061855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:22.078190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35be955-bb89-4af0-b866-75658750ce62 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:22.105011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:22.106273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:22.106772 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:22.107250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] PATH_INFO: `/v3/projects/8ac4035298854afabf3fb41facdfccc9/users/16f6346faeb547ce85746ed6e6caa576/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:22.157081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac4035298854afabf3fb41facdfccc9, user_id=16f6346faeb547ce85746ed6e6caa576, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:22.160150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:22.225717 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1152/2303] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:47:22 2022] PUT /identity/v3/projects/8ac4035298854afabf3fb41facdfccc9/users/16f6346faeb547ce85746ed6e6caa576/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:22.239690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:22.240005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:22.240322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:22.313201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] MFA Rules not processed for user `16f6346faeb547ce85746ed6e6caa576`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:22.324971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:22.469299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1152/2304] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:22.484339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:22.487447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:22.644422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65f9a-4a74-4dc3-83c7-575dbf39f3af admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:22.659581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e1528-6603-42c9-aac6-4c4ca4048f01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:22.660437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1528-6603-42c9-aac6-4c4ca4048f01 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:22.660929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1528-6603-42c9-aac6-4c4ca4048f01 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:22.662449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e1528-6603-42c9-aac6-4c4ca4048f01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:22.672931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1528-6603-42c9-aac6-4c4ca4048f01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:22.674785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e1528-6603-42c9-aac6-4c4ca4048f01 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:22.686867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1153/2305] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:23.242778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:23.244363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:23.430305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb1268-f83b-4198-bd56-7aeaf30ce041 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:23.452555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:23.456467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:23.457150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:23.457648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] PATH_INFO: `/v3/users/d2cdf17fb3be47cda297fc2c805cc573` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:23.467400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2cdf17fb3be47cda297fc2c805cc573)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:23.469649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:23.550917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2cdf17fb3be47cda297fc2c805cc573'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:23.565869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2cdf17fb3be47cda297fc2c805cc573'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:23.566628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2cdf17fb3be47cda297fc2c805cc573'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:23.567237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2cdf17fb3be47cda297fc2c805cc573'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:23.574030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2cdf17fb3be47cda297fc2c805cc573'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:23.594956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1153/2306] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:23 2022] DELETE /identity/v3/users/d2cdf17fb3be47cda297fc2c805cc573 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:23.600124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:23.601739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:23.602779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:23.625214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:23.627403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:23.792602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1154/2307] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:23.807562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:23.810369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:23.918966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4ccf-1388-4793-926c-eb6c2010340d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:23.923266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:23.925105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:23.925390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:23.925730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:23.938778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:23.941221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:23.963900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1154/2308] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:47:24.273510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:24.275284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:24.290678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608797cd-4c28-4cf6-80d9-1d769d9db603 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:24.297063 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:24.299511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:24.300516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:24.301466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] PATH_INFO: `/v3/projects/2e2c86afb03e4ad8afd66ef5ff9a7ec7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:24.303132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2c86afb03e4ad8afd66ef5ff9a7ec7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:24.306139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:24.356846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2c86afb03e4ad8afd66ef5ff9a7ec7', 'request_id': 'req-63e2559f-804b-4649-8c8d-e01e3ac22f45'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:24.365135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2c86afb03e4ad8afd66ef5ff9a7ec7', 'request_id': 'req-63e2559f-804b-4649-8c8d-e01e3ac22f45'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:24.370889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2c86afb03e4ad8afd66ef5ff9a7ec7', 'request_id': 'req-63e2559f-804b-4649-8c8d-e01e3ac22f45'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:24.388874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] The token cache is being invalidate because project 2e2c86afb03e4ad8afd66ef5ff9a7ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:24.389068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:24.390187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-63e2559f-804b-4649-8c8d-e01e3ac22f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:24.391802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1155/2309] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:24 2022] DELETE /identity/v3/projects/2e2c86afb03e4ad8afd66ef5ff9a7ec7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:28.922396 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:28.926158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:28.950040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefcfb0-e75e-4833-b9d5-37bec5c5efe2 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:29.037513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:29.044450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:29.045183 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:29.045699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] PATH_INFO: `/v3/users/e0d919cfe3e0434c9833e4013ad217cc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:29.047585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d919cfe3e0434c9833e4013ad217cc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:29.051917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:29.120285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d919cfe3e0434c9833e4013ad217cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.144319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d919cfe3e0434c9833e4013ad217cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.147699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d919cfe3e0434c9833e4013ad217cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.148326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d919cfe3e0434c9833e4013ad217cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.156612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d919cfe3e0434c9833e4013ad217cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.179941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1155/2310] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:28 2022] DELETE /identity/v3/users/e0d919cfe3e0434c9833e4013ad217cc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:29.181857 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:29.182416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:29.182416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:29.204084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:29.205456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:29.284464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1156/2311] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:29.294349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:29.295871 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:29.367477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ece86-d2bf-4ca4-b866-34b573535f24 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:29.371290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471f08-0cc6-4a45-9c89-141852983322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:29.371961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471f08-0cc6-4a45-9c89-141852983322 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:29.372270 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471f08-0cc6-4a45-9c89-141852983322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:29.372547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471f08-0cc6-4a45-9c89-141852983322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:29.389566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12471f08-0cc6-4a45-9c89-141852983322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:29.390931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12471f08-0cc6-4a45-9c89-141852983322 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:29.404894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1156/2312] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:29.504127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:29.506174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:29.520879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2dcb7-5048-4615-9e19-6d9997266b63 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:29.525622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:29.526840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:29.526840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:29.527034 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] PATH_INFO: `/v3/users/3d73c95e62474cb185bae080248af9da` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:29.528518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d73c95e62474cb185bae080248af9da)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:29.531311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:29.626323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d73c95e62474cb185bae080248af9da'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.636858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d73c95e62474cb185bae080248af9da'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.647674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d73c95e62474cb185bae080248af9da'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.648684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d73c95e62474cb185bae080248af9da'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.667422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d73c95e62474cb185bae080248af9da'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.674541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1157/2313] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:47:29 2022] DELETE /identity/v3/users/3d73c95e62474cb185bae080248af9da => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:29.678986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:29.679429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:29.679626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:29.701935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:29.703283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:29.840187 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1157/2314] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:29.858180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:29.859713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:29.870629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:29.872135 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:29.894029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d2aa-2790-4531-a3e5-a513ba3ceeb6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:29.906120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:29.908459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:29.908459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:29.908459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] PATH_INFO: `/v3/projects/3e0a5b853e6347ffa61e544dddd501a6` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:29.912262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0a5b853e6347ffa61e544dddd501a6)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:29.913765 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:29.981209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0a5b853e6347ffa61e544dddd501a6', 'request_id': 'req-072d5e84-7d1d-416c-872f-5fd0562895e3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:29.984651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789372f-3431-45c9-a0f4-e451be883dc1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:29.993674 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0a5b853e6347ffa61e544dddd501a6', 'request_id': 'req-072d5e84-7d1d-416c-872f-5fd0562895e3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.007564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0a5b853e6347ffa61e544dddd501a6', 'request_id': 'req-072d5e84-7d1d-416c-872f-5fd0562895e3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.030028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] The token cache is being invalidate because project 3e0a5b853e6347ffa61e544dddd501a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:30.030362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.032608 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.048585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1158/2315] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:29 2022] DELETE /identity/v3/projects/3e0a5b853e6347ffa61e544dddd501a6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:47:30.069023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:30.070003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:30.070579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:30.071023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:30.076872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:30.212263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:30.213846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59a185-f05c-4459-9efa-0cdd25ea29c7 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:30.226459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1158/2316] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:30.819574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:30.821296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:30.841143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d5e84-7d1d-416c-872f-5fd0562895e3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:30.847610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:30.848584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:30.849044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:30.849347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] PATH_INFO: `/v3/projects/11d3e88c3ffe43beb17fbfcb2fbf6702` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:30.851085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d3e88c3ffe43beb17fbfcb2fbf6702)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:30.852868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:30.928419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d3e88c3ffe43beb17fbfcb2fbf6702', 'request_id': 'req-584298d7-7bd7-4e2e-9043-245bbb41d3f7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.942044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d3e88c3ffe43beb17fbfcb2fbf6702', 'request_id': 'req-584298d7-7bd7-4e2e-9043-245bbb41d3f7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.948897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d3e88c3ffe43beb17fbfcb2fbf6702', 'request_id': 'req-584298d7-7bd7-4e2e-9043-245bbb41d3f7'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.961202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] The token cache is being invalidate because project 11d3e88c3ffe43beb17fbfcb2fbf6702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:47:30.961496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.962343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-584298d7-7bd7-4e2e-9043-245bbb41d3f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:47:30.964423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1159/2317] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:47:30 2022] DELETE /identity/v3/projects/11d3e88c3ffe43beb17fbfcb2fbf6702 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:47:34.746361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70119a-cb90-4c1a-973a-03c50dc41db3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:34.746894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70119a-cb90-4c1a-973a-03c50dc41db3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:34.746894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70119a-cb90-4c1a-973a-03c50dc41db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:34.766039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8d70119a-cb90-4c1a-973a-03c50dc41db3 None None] MFA Rules not processed for user `298a13924a864f61adf6e4085e569358`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:34.767380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d70119a-cb90-4c1a-973a-03c50dc41db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:34.834369 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1159/2318] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:47:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:38.883971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec0ad-7fdc-4e8e-b950-5af10239cb69 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:38.884585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec0ad-7fdc-4e8e-b950-5af10239cb69 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:38.884751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec0ad-7fdc-4e8e-b950-5af10239cb69 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:38.886140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1160/2319] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:47:38.892231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f7121-45ab-4da7-b7f5-999ee2867465 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:38.892972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f7121-45ab-4da7-b7f5-999ee2867465 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:38.892972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f7121-45ab-4da7-b7f5-999ee2867465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.005779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-183f7121-45ab-4da7-b7f5-999ee2867465 None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:39.008124 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-183f7121-45ab-4da7-b7f5-999ee2867465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:39.124401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1160/2320] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:47:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:39.136341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aec0ad-7fdc-4e8e-b950-5af10239cb69 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:39.138019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f5aec0ad-7fdc-4e8e-b950-5af10239cb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:39.195165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aec0ad-7fdc-4e8e-b950-5af10239cb69 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:39.199295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134cecc-69a8-479b-a54b-f7fa0dded0da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:39.199951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134cecc-69a8-479b-a54b-f7fa0dded0da service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.200271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134cecc-69a8-479b-a54b-f7fa0dded0da service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.200551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134cecc-69a8-479b-a54b-f7fa0dded0da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.212019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134cecc-69a8-479b-a54b-f7fa0dded0da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:39.213482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134cecc-69a8-479b-a54b-f7fa0dded0da service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:39.225305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1161/2321] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:47:39.293820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962e93-012e-40fb-816b-b82032dd9c98 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.294111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962e93-012e-40fb-816b-b82032dd9c98 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.294391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962e93-012e-40fb-816b-b82032dd9c98 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.295758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1161/2322] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:47:39.301493 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5bbea-758b-4fbd-b60a-b0a196cd3c0a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.301852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5bbea-758b-4fbd-b60a-b0a196cd3c0a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.302197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5bbea-758b-4fbd-b60a-b0a196cd3c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.320996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b2c5bbea-758b-4fbd-b60a-b0a196cd3c0a None None] MFA Rules not processed for user `6076a5f71eed4f8fa47d74a79d6fa709`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:39.322205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b2c5bbea-758b-4fbd-b60a-b0a196cd3c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:39.382516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1162/2323] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:47:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:47:39.396800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26962e93-012e-40fb-816b-b82032dd9c98 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:39.398730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-26962e93-012e-40fb-816b-b82032dd9c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:39.414713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26962e93-012e-40fb-816b-b82032dd9c98 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:39.418514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805afc40-237a-4fb8-a665-045dc0542d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:39.419154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805afc40-237a-4fb8-a665-045dc0542d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.419434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805afc40-237a-4fb8-a665-045dc0542d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.419704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805afc40-237a-4fb8-a665-045dc0542d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.427679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805afc40-237a-4fb8-a665-045dc0542d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:39.429263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805afc40-237a-4fb8-a665-045dc0542d7f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:39.440398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1162/2324] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:39.541481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30615689-45df-41d0-9b5d-f8a706b721b9 None None] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.541854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30615689-45df-41d0-9b5d-f8a706b721b9 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.542041 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30615689-45df-41d0-9b5d-f8a706b721b9 None None] PATH_INFO: `/` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.543490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1163/2325] 10.222.0.73 () {54 vars in 921 bytes} [Thu May 12 17:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 12 17:47:39.550678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a72cf1-b84b-4ece-bf79-e437891cd64f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.550968 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a72cf1-b84b-4ece-bf79-e437891cd64f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.551271 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a72cf1-b84b-4ece-bf79-e437891cd64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.575191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b9a72cf1-b84b-4ece-bf79-e437891cd64f None None] MFA Rules not processed for user `62c3b98740d14ab49d91783086150e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:39.580960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9a72cf1-b84b-4ece-bf79-e437891cd64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:39.598374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1163/2326] 10.222.0.73 () {58 vars in 1018 bytes} [Thu May 12 17:47:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:39.609225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30615689-45df-41d0-9b5d-f8a706b721b9 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:39.627268 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30615689-45df-41d0-9b5d-f8a706b721b9 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:39.638890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:39.639554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:39.639839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:39.640119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:39.656691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:39.658150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:39.671232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1164/2327] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:47:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:47:49.758788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:49.759264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:49.759377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:49.809221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:47:49.810491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:49.876322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1164/2328] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:47:49.900676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:47:49.903061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:47:50.023468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b606a-1170-456e-ae81-2c119a85aecb service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:47:50.029934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:47:50.030631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:47:50.030920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:47:50.031355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:47:50.042478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:47:50.043733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:47:50.059207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1165/2329] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:47:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:48:08.285218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:08.286871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:08.382308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16e8d-e856-4954-9b96-a94550783444 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:08.388371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:08.389018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:08.389329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:08.389608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:08.391379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:08.403975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:08.405584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4dea-f053-46dd-bf7b-ee71a7db0f59 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:08.416073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1165/2330] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:48:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:48:11.602721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:11.604213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:11.686811 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3b39c-b266-4e70-aacf-6fa31698f5b4 service swift] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:11.692494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:11.693619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:11.694080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:11.694508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] PATH_INFO: `/v3/users/16f6346faeb547ce85746ed6e6caa576` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:11.695874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f6346faeb547ce85746ed6e6caa576)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:11.697978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:11.757644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f6346faeb547ce85746ed6e6caa576'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:11.770530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f6346faeb547ce85746ed6e6caa576'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:11.771225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f6346faeb547ce85746ed6e6caa576'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:11.771856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f6346faeb547ce85746ed6e6caa576'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:11.778689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f6346faeb547ce85746ed6e6caa576'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:11.791685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1166/2331] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:48:11 2022] DELETE /identity/v3/users/16f6346faeb547ce85746ed6e6caa576 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:48:11.796225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:11.796709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:11.796925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:11.822222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:11.823433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:11.885259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1166/2332] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:48:11.897974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:11.899516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:11.961939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11e855-5e3a-40af-ac9f-3174d2460edb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:11.965661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c877f58-fee7-405c-b652-829500998e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:11.966322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877f58-fee7-405c-b652-829500998e31 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:11.966609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877f58-fee7-405c-b652-829500998e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:11.966888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877f58-fee7-405c-b652-829500998e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:11.978822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c877f58-fee7-405c-b652-829500998e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:11.980515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c877f58-fee7-405c-b652-829500998e31 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:11.992986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1167/2333] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:48:12.305235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.306652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:12.320340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15512d-d40e-4bdf-8427-c912af92c13c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.325878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.327039 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.327551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.328365 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] PATH_INFO: `/v3/projects/8ac4035298854afabf3fb41facdfccc9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.330292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac4035298854afabf3fb41facdfccc9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.332958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.406373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac4035298854afabf3fb41facdfccc9', 'request_id': 'req-e91180c2-89c2-452b-a428-6d20022bcda9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:12.417504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac4035298854afabf3fb41facdfccc9', 'request_id': 'req-e91180c2-89c2-452b-a428-6d20022bcda9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:12.445308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac4035298854afabf3fb41facdfccc9', 'request_id': 'req-e91180c2-89c2-452b-a428-6d20022bcda9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:12.465368 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] The token cache is being invalidate because project 8ac4035298854afabf3fb41facdfccc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:48:12.465573 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:12.466548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91180c2-89c2-452b-a428-6d20022bcda9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:12.468770 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1167/2334] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:48:12 2022] DELETE /identity/v3/projects/8ac4035298854afabf3fb41facdfccc9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:48:12.476230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.476736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.477154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.501891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:12.503971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:12.582353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1168/2335] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:48:12.590824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.591351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.591958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.616550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:12.618038 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:12.626505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1168/2336] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:48:12.631830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.641697 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386afb-ebb5-4b39-bb12-221998edb121 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.647436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.648103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.648554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.648838 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.649582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:48:12.649892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.651480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.663409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1169/2337] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:48:12.667544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.677477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f947351-4424-41d6-a96f-f6f2f91edd4f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.682290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.682940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.683220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.683492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.684189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.686086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.713922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1169/2338] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:48:12 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:48:12.715857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.726284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023068a-0078-40fc-ba53-6c1fef721ea7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.780988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.781677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.781956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.782223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.782935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.785620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.806294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1170/2339] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:48:12 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:48:12.810165 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.820714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd737c2-9409-44f8-baf6-56d57a369e11 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.825250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.825902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.826180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.826451 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.827301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.828907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.839649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1170/2340] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:48:12.843520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.853885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d084-8272-4df1-8d4d-47ca99239704 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.858651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ab63c-66c4-4f73-b9bb-c398013d947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.859295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab63c-66c4-4f73-b9bb-c398013d947f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.859578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab63c-66c4-4f73-b9bb-c398013d947f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.859855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab63c-66c4-4f73-b9bb-c398013d947f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.860605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ab63c-66c4-4f73-b9bb-c398013d947f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.862797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ab63c-66c4-4f73-b9bb-c398013d947f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.872474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1171/2341] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:48:12.876743 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:12.887015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3261a-1c3c-461e-bd02-bc4474eef6a6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:12.891603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:12.892533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.892829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.894235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] PATH_INFO: `/v3/projects/9ee4b44f93824e26ae44458f1b1dd654/users/ef70afb0038a4cebaeb18571da14e7d5/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.910834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee4b44f93824e26ae44458f1b1dd654, user_id=ef70afb0038a4cebaeb18571da14e7d5, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:12.914149 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:12.928527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1171/2342] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:48:12 2022] PUT /identity/v3/projects/9ee4b44f93824e26ae44458f1b1dd654/users/ef70afb0038a4cebaeb18571da14e7d5/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:48:12.939417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:12.939738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:12.940768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:12.968622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] MFA Rules not processed for user `ef70afb0038a4cebaeb18571da14e7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:12.969929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:13.039981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1172/2343] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:48:13.052456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:13.054231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:13.133012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183e18-16ac-468c-8978-4833771d84c3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:13.136611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc5e11-c2b3-46ce-9b77-6501b863cfdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:13.137252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5e11-c2b3-46ce-9b77-6501b863cfdd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:13.137530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5e11-c2b3-46ce-9b77-6501b863cfdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:13.137800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc5e11-c2b3-46ce-9b77-6501b863cfdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:13.145947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc5e11-c2b3-46ce-9b77-6501b863cfdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:13.147358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc5e11-c2b3-46ce-9b77-6501b863cfdd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:13.155869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1172/2344] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:48:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:48:17.763314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:17.822458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b249a3-1b00-45fe-b6d6-18d92e604899 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:17.826981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:17.827834 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:17.828115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:17.828618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] PATH_INFO: `/v3/users/ef70afb0038a4cebaeb18571da14e7d5` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:17.829798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef70afb0038a4cebaeb18571da14e7d5)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:17.831800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:17.877293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef70afb0038a4cebaeb18571da14e7d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:17.888499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef70afb0038a4cebaeb18571da14e7d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:17.889181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef70afb0038a4cebaeb18571da14e7d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:17.889795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef70afb0038a4cebaeb18571da14e7d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:17.896832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef70afb0038a4cebaeb18571da14e7d5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:17.910328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1173/2345] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:48:17 2022] DELETE /identity/v3/users/ef70afb0038a4cebaeb18571da14e7d5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:48:17.915294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:17.915740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:17.915881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:17.934129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:17.935313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:17.992673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1173/2346] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:48:18.003336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:18.004758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:18.058830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748d265-046a-4403-8dea-91d6beacf9be admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:18.062438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233f6e3-657f-41c9-b52d-0a66c5c2b985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:18.063071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233f6e3-657f-41c9-b52d-0a66c5c2b985 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:18.063357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233f6e3-657f-41c9-b52d-0a66c5c2b985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:18.063634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233f6e3-657f-41c9-b52d-0a66c5c2b985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:18.072657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233f6e3-657f-41c9-b52d-0a66c5c2b985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:18.074074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233f6e3-657f-41c9-b52d-0a66c5c2b985 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:18.083308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1174/2347] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:48:18.343488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:18.345049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:18.359841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae354581-d1ae-4ac8-841e-9e2b39d6f94a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:18.371961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:18.372932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:18.373210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:18.373481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] PATH_INFO: `/v3/projects/9ee4b44f93824e26ae44458f1b1dd654` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:18.374670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee4b44f93824e26ae44458f1b1dd654)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:18.377302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:18.436078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee4b44f93824e26ae44458f1b1dd654', 'request_id': 'req-13cd87dc-d3a4-402a-9023-b25d30a701f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:18.443479 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee4b44f93824e26ae44458f1b1dd654', 'request_id': 'req-13cd87dc-d3a4-402a-9023-b25d30a701f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:18.450966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee4b44f93824e26ae44458f1b1dd654', 'request_id': 'req-13cd87dc-d3a4-402a-9023-b25d30a701f6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:18.466838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] The token cache is being invalidate because project 9ee4b44f93824e26ae44458f1b1dd654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:48:18.467233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:18.467990 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13cd87dc-d3a4-402a-9023-b25d30a701f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:48:18.470207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1174/2348] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:48:18 2022] DELETE /identity/v3/projects/9ee4b44f93824e26ae44458f1b1dd654 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:48:21.637891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:21.638392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:21.638490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:21.656042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] MFA Rules not processed for user `5ad6490c8b8249b1b2abcb0f359330df`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:21.657257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:21.709301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1175/2349] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:48:23.484736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c5fd0-03f1-4707-8bd2-3ef343684a75 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:23.485292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c5fd0-03f1-4707-8bd2-3ef343684a75 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:23.485292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c5fd0-03f1-4707-8bd2-3ef343684a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:23.504768 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bd3c5fd0-03f1-4707-8bd2-3ef343684a75 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:23.506281 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bd3c5fd0-03f1-4707-8bd2-3ef343684a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:23.563057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1175/2350] 10.222.0.73 () {58 vars in 967 bytes} [Thu May 12 17:48:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:48:23.575432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:48:23.576857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:23.649265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7001f-8953-49b7-9dcb-07856e2ac55b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:48:23.654271 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74e061-547f-463a-8593-e252ff574a9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:48:23.654922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74e061-547f-463a-8593-e252ff574a9d service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:23.655226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74e061-547f-463a-8593-e252ff574a9d service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:23.655483 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74e061-547f-463a-8593-e252ff574a9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:23.664412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74e061-547f-463a-8593-e252ff574a9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:48:23.665987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74e061-547f-463a-8593-e252ff574a9d service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:48:23.673860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1176/2351] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:48:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:48:55.215088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:48:55.215645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:48:55.215645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:48:55.235103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] MFA Rules not processed for user `1cc16878bc6a445f8a66dfcdc54a7e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:48:55.236603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:48:55.297415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1176/2352] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:48:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:21.010374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:21.010980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:21.010980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:21.028912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] MFA Rules not processed for user `7a2b51125e9645dc9454042f49ff3659`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:21.030094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:21.083484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1177/2353] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:23.733655 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:23.735161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:23.810981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e98db-1670-4cef-863c-fa5b3dbdeb79 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:23.815527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:23.816513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:23.816860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:23.817230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] PATH_INFO: `/v3/users/5ad6490c8b8249b1b2abcb0f359330df` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:23.818455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad6490c8b8249b1b2abcb0f359330df)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:23.821126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:23.875909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad6490c8b8249b1b2abcb0f359330df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:23.884574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad6490c8b8249b1b2abcb0f359330df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:23.892311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad6490c8b8249b1b2abcb0f359330df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:23.893099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad6490c8b8249b1b2abcb0f359330df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:23.906177 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-862a556e-6214-4276-b5ff-07d5ba49cecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad6490c8b8249b1b2abcb0f359330df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:23.915421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1177/2354] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:49:23 2022] DELETE /identity/v3/users/5ad6490c8b8249b1b2abcb0f359330df => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:49:24.485131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:24.498708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b4bf7-9229-469f-94d9-df5b9a42d1fe None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:24.564308 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:24.565157 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.565445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.565719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] PATH_INFO: `/v3/projects/7ef859014e4d4ff79fb7958511f20d11` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.566981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef859014e4d4ff79fb7958511f20d11)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:24.568913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:24.616820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef859014e4d4ff79fb7958511f20d11', 'request_id': 'req-7d157b30-471f-43fa-8cdc-356cf313dfa2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:24.625401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef859014e4d4ff79fb7958511f20d11', 'request_id': 'req-7d157b30-471f-43fa-8cdc-356cf313dfa2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:24.631032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef859014e4d4ff79fb7958511f20d11', 'request_id': 'req-7d157b30-471f-43fa-8cdc-356cf313dfa2'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:24.642235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] The token cache is being invalidate because project 7ef859014e4d4ff79fb7958511f20d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:49:24.642524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:24.643160 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7d157b30-471f-43fa-8cdc-356cf313dfa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:49:24.644738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1178/2355] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:49:24 2022] DELETE /identity/v3/projects/7ef859014e4d4ff79fb7958511f20d11 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:49:24.650332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.650622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.650900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.668178 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:24.669440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:24.731375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1178/2356] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:24.739346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.739642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.739924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.761013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:24.762198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:24.770629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1179/2357] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:24.774798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:24.784191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa7ed-980d-46ac-a7b9-9980777586fe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:24.788715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:24.789359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.789632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.789912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.790622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:49:24.790922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:24.792532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:24.804967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1179/2358] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:49:24.809351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:24.818566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57911bb-9ae7-4b59-881a-2ba11d972a72 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:24.823998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:24.824675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.824959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.825236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.825935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:24.827821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:24.852777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1180/2359] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:49:24 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:49:24.856819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:24.867659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac4ee-f850-4613-aa57-f4051444e136 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:24.923357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:24.924017 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.924362 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.924640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.925329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:24.927227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:24.945615 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1180/2360] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:49:24 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:49:24.950858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:24.961687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864d81-75fb-41d8-8fda-c750e6fa7a7f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:24.966289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:24.966934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:24.967306 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:24.967501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:24.968233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:24.969872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:24.980708 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1181/2361] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:49:24.984636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:24.994771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e5fa9-79dc-48e0-95a0-b4433daa83a7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:24.999370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1f6dc-5def-4100-9920-c073d84d891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:25.000019 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f6dc-5def-4100-9920-c073d84d891f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:25.000331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f6dc-5def-4100-9920-c073d84d891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:25.000633 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f6dc-5def-4100-9920-c073d84d891f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:25.001597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1f6dc-5def-4100-9920-c073d84d891f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:25.003168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1f6dc-5def-4100-9920-c073d84d891f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:25.012756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1181/2362] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:49:25.016681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:25.026756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e3168-71d0-44a4-a9a2-db1230315837 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:25.031185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:25.032046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:25.032356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:25.032631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] PATH_INFO: `/v3/projects/a4d7ebeafe8140b3b8332059258a83eb/users/adbaa9b8b7e2447da5eef8f32aff22a1/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:25.047191 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d7ebeafe8140b3b8332059258a83eb, user_id=adbaa9b8b7e2447da5eef8f32aff22a1, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:25.049874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:25.062637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1182/2363] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:49:25 2022] PUT /identity/v3/projects/a4d7ebeafe8140b3b8332059258a83eb/users/adbaa9b8b7e2447da5eef8f32aff22a1/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:49:25.068148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd7502-9966-40c7-b628-f47fc56d4bb0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:25.068484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd7502-9966-40c7-b628-f47fc56d4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:25.068765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd7502-9966-40c7-b628-f47fc56d4bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:25.086854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-facd7502-9966-40c7-b628-f47fc56d4bb0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:25.088043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-facd7502-9966-40c7-b628-f47fc56d4bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:25.143825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1182/2364] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:25.153779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:25.155501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:25.213976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1ad96-69fb-4aed-9896-7bebb1ab0eb7 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:25.217591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:25.218226 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:25.218506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:25.218781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:25.227676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:25.229115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:25.238102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1183/2365] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:49:30.020326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.020909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.020909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.050046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] MFA Rules not processed for user `adbaa9b8b7e2447da5eef8f32aff22a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:30.051755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:30.128406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1183/2366] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:30.133689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:30.146220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f3b0-c221-436a-8d48-f515aefd3a8a service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:30.153678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:30.154566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.154918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.155303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.156246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:30.158698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:30.195357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1184/2367] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:49:30 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:49:30.197231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:30.212496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950fe4e-5d5d-4a19-a947-360e9dfdc343 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:30.274645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:30.275367 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.275653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.275930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.276647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:30.278577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:30.302797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1184/2368] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:49:30 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:49:30.304145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:30.315328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1649-486a-4631-9850-60fdcdbf634f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:30.320804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:30.321398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.321828 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.322110 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.322847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:30.329364 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:30.352741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:30.356097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1185/2369] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:49:30.364873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda350d-5a92-4639-82c0-67279b83baa9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:30.370130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:30.371033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.371460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.371774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] PATH_INFO: `/v3/projects/d7c90277db3d4f89abd17dc952d72400/users/22beb108fd7d4479b827cd11e2383774/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.388931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c90277db3d4f89abd17dc952d72400, user_id=22beb108fd7d4479b827cd11e2383774, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:30.392046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:30.407346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1185/2370] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:49:30 2022] PUT /identity/v3/projects/d7c90277db3d4f89abd17dc952d72400/users/22beb108fd7d4479b827cd11e2383774/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:49:30.411064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:30.421509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a28125-d100-4f37-8cb9-e65816dfa8b5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:30.484432 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9833b0-e51c-4d26-b80c-dcf02e00e93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:30.485098 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9833b0-e51c-4d26-b80c-dcf02e00e93c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.485378 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9833b0-e51c-4d26-b80c-dcf02e00e93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.485654 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9833b0-e51c-4d26-b80c-dcf02e00e93c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.486374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9833b0-e51c-4d26-b80c-dcf02e00e93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:30.487938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9833b0-e51c-4d26-b80c-dcf02e00e93c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:30.501776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1186/2371] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:49:30.502947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:30.512804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e4a94-756c-4145-aff4-b1d4746c460b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:30.517263 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3938660-12dd-4498-9694-b6deef960195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:30.518151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3938660-12dd-4498-9694-b6deef960195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:30.518423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3938660-12dd-4498-9694-b6deef960195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:30.518686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3938660-12dd-4498-9694-b6deef960195 admin admin] PATH_INFO: `/v3/projects/d7c90277db3d4f89abd17dc952d72400/users/22beb108fd7d4479b827cd11e2383774/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:30.520664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3938660-12dd-4498-9694-b6deef960195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c90277db3d4f89abd17dc952d72400, user_id=22beb108fd7d4479b827cd11e2383774, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:30.523373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3938660-12dd-4498-9694-b6deef960195 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:30.533874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1186/2372] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:49:30 2022] PUT /identity/v3/projects/d7c90277db3d4f89abd17dc952d72400/users/22beb108fd7d4479b827cd11e2383774/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:49:36.248323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:36.250375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:36.250821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:36.271497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] MFA Rules not processed for user `298a13924a864f61adf6e4085e569358`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:36.273866 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:36.435983 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1187/2373] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:36.815174 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdb433-b178-4d80-a9a2-87394bfd9ff6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:36.816731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdb433-b178-4d80-a9a2-87394bfd9ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:36.817269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdb433-b178-4d80-a9a2-87394bfd9ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:36.858822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cacdb433-b178-4d80-a9a2-87394bfd9ff6 None None] MFA Rules not processed for user `22beb108fd7d4479b827cd11e2383774`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:36.860746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cacdb433-b178-4d80-a9a2-87394bfd9ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:36.963391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1187/2374] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:36.980431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:36.982197 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:37.120545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1fcc4-164d-49dd-a804-1baf1785e201 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:37.126300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:37.127233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:37.127532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:37.127793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:37.186399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:37.188618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:37.198307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1188/2375] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:49:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:49:54.759960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:54.760565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:54.760565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:54.778737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] MFA Rules not processed for user `adbaa9b8b7e2447da5eef8f32aff22a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:49:54.779935 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:54.786767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1188/2376] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:49:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:49:54.823196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:54.824690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:54.878791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ffb9c-25b0-4fbe-bc82-27d6a1f805c4 service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:54.882276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:54.882910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:54.883192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:54.883468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:54.885244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:54.898504 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:54.900048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a414f-a1b9-426e-9310-2996459197a5 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:54.911014 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1189/2377] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:49:57.823480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:49:57.824865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:57.884018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6e9c4-9b4b-46fc-8f8f-70530ebe67d4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:49:57.887578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:49:57.888266 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:49:57.888587 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:49:57.889032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:49:57.890912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:49:57.906777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:49:57.908218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:49:57.921243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1189/2378] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:49:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:50:06.806879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:06.807421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:06.807421 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:06.825016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] MFA Rules not processed for user `1cc16878bc6a445f8a66dfcdc54a7e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:06.826210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:06.879956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1190/2379] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:07.478090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:07.479551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:07.547070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d47fe-3f6a-41ee-b6f3-0e74107f1f8e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:07.552000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:07.552672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:07.552956 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:07.553226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:07.555774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:07.566524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:07.568083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:07.578628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1190/2380] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:10.282886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:10.301411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84131b8-dcaf-4c1b-85bb-9dad0d4aaa0a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:10.305349 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:10.306260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:10.306713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:10.307149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:10.309284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:10.321184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:10.323462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:10.336581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1191/2381] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:13.338947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:13.340377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:13.355939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f675a3d-f68c-4ed3-9f28-5802584bfdd7 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:13.361400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:13.362041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:13.362323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:13.362595 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:13.364454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:13.375219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:13.376832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:13.386752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1191/2382] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:13.767743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:13.776477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfb7cc-bf2c-46a0-b942-c3d93925d742 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:13.780042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:13.780706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:13.780988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:13.781264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:13.790997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:13.792435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:13.801590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1192/2383] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:17.871741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:17.883548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033994f2-2b49-4d13-8259-c72b2fd539cc service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:17.888950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:17.889579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:17.889862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:17.890134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:17.891926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:17.949844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:17.951424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:17.960497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1192/2384] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:20.620640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:20.622120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:20.634775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618e6a-9418-4b6a-943a-e61df2dfc45d service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:20.638338 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:20.638972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:20.639257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:20.639535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:20.641290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:20.653242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:20.654672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:20.664635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1193/2385] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:26.811512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:26.824899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921afc8-15d3-460e-816e-0c792e913b30 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:26.831346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:26.832103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:26.832437 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:26.832712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:26.844252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:26.860975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:26.863202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:26.876454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1193/2386] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:40.369509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:40.376869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:40.387592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb90378-e6f3-42f9-9cbc-efbe9190604f service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:40.389012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696531-8486-40bb-8652-d65808bc03ce service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:40.392818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:40.393227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:40.393791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:40.393927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:40.394145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:40.394461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:40.394592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:40.394931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:40.396381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:40.396810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:40.408574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:40.410404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ee746-bf30-4d1d-afd7-ba7dbfe56c51 service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:40.423599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1194/2387] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:40.466196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:40.467761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:40.477410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1194/2388] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:50:41.257951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:41.258317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:41.258670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:41.278264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:41.279478 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:41.336455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1195/2389] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:41.396458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:41.397867 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:41.478222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00041d8b-3c1e-4a74-aecd-425c26fe9f52 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:41.484068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:41.484747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:41.485411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:41.485848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:41.500669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:41.502265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:41.511802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1195/2390] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:50:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:50:44.853044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:44.857283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:44.876585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033395d8-8ba8-45e9-ad15-b5cb5fe78479 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:44.883663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:44.884813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:44.885153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:44.885488 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] PATH_INFO: `/v3/users/298a13924a864f61adf6e4085e569358` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:44.887272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298a13924a864f61adf6e4085e569358)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:44.889504 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:44.952624 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298a13924a864f61adf6e4085e569358'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:44.966171 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298a13924a864f61adf6e4085e569358'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:44.966920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298a13924a864f61adf6e4085e569358'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:44.967537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298a13924a864f61adf6e4085e569358'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:44.976616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-fdb9cf92-2fc5-4653-9fdb-4532f68843b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298a13924a864f61adf6e4085e569358'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:44.996691 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1196/2391] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:44 2022] DELETE /identity/v3/users/298a13924a864f61adf6e4085e569358 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:45.275171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.301932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900fb71-f6bc-4b2f-838a-3da1eace1ba4 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.405916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.406808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.407082 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.407351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] PATH_INFO: `/v3/projects/202db80dae2446e2b9cd1124b2925ced` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.408692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=202db80dae2446e2b9cd1124b2925ced)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.410896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:45.463267 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202db80dae2446e2b9cd1124b2925ced', 'request_id': 'req-385a11cc-79d9-4463-b74b-d4f8a55af22b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:45.471139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202db80dae2446e2b9cd1124b2925ced', 'request_id': 'req-385a11cc-79d9-4463-b74b-d4f8a55af22b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:45.476872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202db80dae2446e2b9cd1124b2925ced', 'request_id': 'req-385a11cc-79d9-4463-b74b-d4f8a55af22b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:45.491176 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] The token cache is being invalidate because project 202db80dae2446e2b9cd1124b2925ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:50:45.491526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:45.492250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-385a11cc-79d9-4463-b74b-d4f8a55af22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:45.493851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1196/2392] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:50:45 2022] DELETE /identity/v3/projects/202db80dae2446e2b9cd1124b2925ced => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:50:45.506371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.506905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.507358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.527752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:45.529447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:45.596209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1197/2393] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:45.602611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.603604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.603890 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.631110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:45.632612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:45.641845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1197/2394] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:45.646427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.657434 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593028-4864-48f1-8497-69aa4d362fc1 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.664062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.664863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.665256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.665602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.666454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:50:45.666820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.668758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:45.687996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.697703 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1198/2395] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:50:45.698274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796534-8457-4f8f-85b4-c21312ea19f5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.704414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.705248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.705636 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.706015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.706973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.709086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:45.748134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1198/2396] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:50:45 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:45.752044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.767070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3b484-75e7-4a42-b2ed-935227553285 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.827159 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.828032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.828467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.828832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.829683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.831889 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:45.857406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1199/2397] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:50:45 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:45.861602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.875737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd91229-38eb-4202-a95c-52377fc3db81 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.881372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.882042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.882314 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.882587 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.883325 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.884958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:45.898998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1199/2398] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:45.903217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.916342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976dcac-ed98-4ea5-8954-0d33110018cd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.922661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e5bf4-7535-4eda-85eb-960f8d193d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.923738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e5bf4-7535-4eda-85eb-960f8d193d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.924465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e5bf4-7535-4eda-85eb-960f8d193d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.924936 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e5bf4-7535-4eda-85eb-960f8d193d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.925935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e5bf4-7535-4eda-85eb-960f8d193d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.927701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e5bf4-7535-4eda-85eb-960f8d193d8a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:45.938566 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1200/2399] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:45.943160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:45.955393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defa618-a54c-4d64-8b87-4d11f26c49b3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:45.960347 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:45.961384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:45.961730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:45.962069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] PATH_INFO: `/v3/projects/922bc2b412de438b966629f235764d45/users/6648c697ab9c4d299bef56888badcc20/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:45.985693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=922bc2b412de438b966629f235764d45, user_id=6648c697ab9c4d299bef56888badcc20, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:45.988394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.008452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1200/2400] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:45 2022] PUT /identity/v3/projects/922bc2b412de438b966629f235764d45/users/6648c697ab9c4d299bef56888badcc20/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:46.019709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.020089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.020468 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.053977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] MFA Rules not processed for user `6648c697ab9c4d299bef56888badcc20`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:46.055301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:46.135087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1201/2401] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:46.139695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.217408 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27ef55-efe3-46b7-a41f-7a0d2aa41f6e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.225650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.226446 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.226790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.227163 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.227950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.230954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.260218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1201/2402] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:50:46 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:50:46.264242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.275761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddd853-e5c6-4cbb-9146-1f00e0050d74 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.340662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.341342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.341625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.341897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.342713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.344787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.375123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1202/2403] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:50:46 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:46.382003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.404215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518a888-be25-44d5-82b9-92efab24a2f7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.419681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.420404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.420738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.421050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.421787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.423547 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.438800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1202/2404] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:46.442785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.461307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c4806-6452-43d7-9821-477cbbe151f3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.466997 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.468019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.468327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.468606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] PATH_INFO: `/v3/projects/e09880d6af0f45da89a4fbe2d8fdaa33/users/461d54b640654e6197c0caf7d8b840f8/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.487302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09880d6af0f45da89a4fbe2d8fdaa33, user_id=461d54b640654e6197c0caf7d8b840f8, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.490178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.508545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1203/2405] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:46 2022] PUT /identity/v3/projects/e09880d6af0f45da89a4fbe2d8fdaa33/users/461d54b640654e6197c0caf7d8b840f8/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:50:46.513541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.524359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de3e65-dd89-4703-9e76-cf7f90e123ce admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.590448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bff24d-7796-421d-8430-33195019a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.591122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bff24d-7796-421d-8430-33195019a06d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.591401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bff24d-7796-421d-8430-33195019a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.591670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bff24d-7796-421d-8430-33195019a06d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.592438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bff24d-7796-421d-8430-33195019a06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.594132 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bff24d-7796-421d-8430-33195019a06d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.608902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1203/2406] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:46.613136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.632365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2327a5-6aab-4c47-80b3-ed7587c91c17 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.637384 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.638399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.638787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.639167 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] PATH_INFO: `/v3/projects/e09880d6af0f45da89a4fbe2d8fdaa33/users/461d54b640654e6197c0caf7d8b840f8/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.641445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09880d6af0f45da89a4fbe2d8fdaa33, user_id=461d54b640654e6197c0caf7d8b840f8, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.646008 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.657295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1204/2407] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:46 2022] PUT /identity/v3/projects/e09880d6af0f45da89a4fbe2d8fdaa33/users/461d54b640654e6197c0caf7d8b840f8/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:46.669542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.669751 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.670134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.700933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] MFA Rules not processed for user `461d54b640654e6197c0caf7d8b840f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:46.702359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:46.788593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1204/2408] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:46.797532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.799277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:46.861418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602db23-eb5b-49ca-92d6-848afd17b193 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.866199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.866917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.867267 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.867605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:46.877798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:46.879242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:46.891348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1205/2409] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:46.979249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:46.989638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008fea1-3d3f-48be-8ad4-dcc4dc1d0d9a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:46.993452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ed4be-3dd1-4b1b-8788-755f6eb9065f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:46.994110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed4be-3dd1-4b1b-8788-755f6eb9065f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:46.994402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed4be-3dd1-4b1b-8788-755f6eb9065f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:46.994662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed4be-3dd1-4b1b-8788-755f6eb9065f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:47.063658 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ed4be-3dd1-4b1b-8788-755f6eb9065f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:47.065403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ed4be-3dd1-4b1b-8788-755f6eb9065f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:47.079985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1205/2410] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:49.889090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:49.958248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1bfc5-ffd0-4c44-b841-6685723f4e87 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:49.965628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:49.966517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:49.966803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:49.967072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] PATH_INFO: `/v3/users/6648c697ab9c4d299bef56888badcc20` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:49.968412 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6648c697ab9c4d299bef56888badcc20)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:49.970479 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:50.069944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6648c697ab9c4d299bef56888badcc20'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.082184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6648c697ab9c4d299bef56888badcc20'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.082931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6648c697ab9c4d299bef56888badcc20'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.084785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6648c697ab9c4d299bef56888badcc20'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.092721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6648c697ab9c4d299bef56888badcc20'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.115480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1206/2411] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:49 2022] DELETE /identity/v3/users/6648c697ab9c4d299bef56888badcc20 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:50.119793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:50.120381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:50.120381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:50.140887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:50.142114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:50.208282 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1206/2412] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:50:50.218645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:50.220134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:50.288299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000d82-1505-4f99-bb44-c6ef3ede04b0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:50.293020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:50.293675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:50.293962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:50.294237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:50.309836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:50.311321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:50.335569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1207/2413] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:50.643354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:50.645090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:50.661690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bde3d4-1970-4bf4-82c9-f7745a4da4cf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:50.667934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:50.668851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:50.669131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:50.669406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] PATH_INFO: `/v3/projects/922bc2b412de438b966629f235764d45` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:50.670647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922bc2b412de438b966629f235764d45)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:50.673696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:50.726645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922bc2b412de438b966629f235764d45', 'request_id': 'req-96a0c39d-7391-407d-b783-29ab5a4c07b5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.733173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922bc2b412de438b966629f235764d45', 'request_id': 'req-96a0c39d-7391-407d-b783-29ab5a4c07b5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.738589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922bc2b412de438b966629f235764d45', 'request_id': 'req-96a0c39d-7391-407d-b783-29ab5a4c07b5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.751726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] The token cache is being invalidate because project 922bc2b412de438b966629f235764d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:50:50.752000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.752904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.754445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1207/2414] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:50:50 2022] DELETE /identity/v3/projects/922bc2b412de438b966629f235764d45 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:50.758296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:50.769463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acff8e-b9c7-4b73-a17d-c155e5f44435 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:50.829619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:50.830593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:50.830878 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:50.831150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] PATH_INFO: `/v3/users/461d54b640654e6197c0caf7d8b840f8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:50.832363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=461d54b640654e6197c0caf7d8b840f8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:50.834294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:50.885606 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461d54b640654e6197c0caf7d8b840f8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.898477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461d54b640654e6197c0caf7d8b840f8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.899160 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461d54b640654e6197c0caf7d8b840f8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.899745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461d54b640654e6197c0caf7d8b840f8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.906874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-97827b8c-a819-4fcb-91d2-3fca17520af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461d54b640654e6197c0caf7d8b840f8'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:50.919366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1208/2415] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:50 2022] DELETE /identity/v3/users/461d54b640654e6197c0caf7d8b840f8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:51.185671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.188053 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:51.202969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c39d-7391-407d-b783-29ab5a4c07b5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.260133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.260941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.261222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.261489 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] PATH_INFO: `/v3/projects/e09880d6af0f45da89a4fbe2d8fdaa33` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.262676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09880d6af0f45da89a4fbe2d8fdaa33)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.264599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.316693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09880d6af0f45da89a4fbe2d8fdaa33', 'request_id': 'req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:51.322758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09880d6af0f45da89a4fbe2d8fdaa33', 'request_id': 'req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:51.330018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09880d6af0f45da89a4fbe2d8fdaa33', 'request_id': 'req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:51.343954 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] The token cache is being invalidate because project e09880d6af0f45da89a4fbe2d8fdaa33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:50:51.344274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:51.344889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f5db9b4-cb2b-43b4-b417-53df54a7e5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:51.346513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1208/2416] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:50:51 2022] DELETE /identity/v3/projects/e09880d6af0f45da89a4fbe2d8fdaa33 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:51.352947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.353312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.353681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.372573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:51.373768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:51.431762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1209/2417] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:51.439403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.439684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.439955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.457946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:51.459135 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:51.467391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1209/2418] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:51.471742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.481922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f70d29-3d59-49dc-8ab9-2f698d886d40 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.486404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.487043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.487325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.487596 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.488352 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:50:51.488653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.490214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.501666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1210/2419] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:50:51.506148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.515838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162a98d-cb35-4c53-82ba-b3f5f3d08861 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.520440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.521094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.521378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.521651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.522384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.524330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.552550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1210/2420] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:51.556595 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.567181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de06192-02a3-4c94-8581-c9d8eb5fb284 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.624443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.625109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.625386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.625657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.626350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.628526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.648509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1211/2421] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:51.653341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.663228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53673a6f-3ecb-4248-a88d-fd5c4138a048 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.667714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.668390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.668686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.668936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.669654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.671229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.681552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1211/2422] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:51.685603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.695964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117119-f550-4000-92d7-79db314fab07 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.701473 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003fa59-66a3-4cfb-86a1-2508b8f90fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.702124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003fa59-66a3-4cfb-86a1-2508b8f90fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.702406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003fa59-66a3-4cfb-86a1-2508b8f90fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.702686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003fa59-66a3-4cfb-86a1-2508b8f90fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.703404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003fa59-66a3-4cfb-86a1-2508b8f90fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.705006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003fa59-66a3-4cfb-86a1-2508b8f90fe1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.718481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1212/2423] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:51.719638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.729525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17511644-bdb7-4c38-b883-c8bce87705d8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.734228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.735102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.735378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.735650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] PATH_INFO: `/v3/projects/946d307c3e8044f7950a3a492d47edbd/users/3e730ba0339444f5995c0ded7adeee59/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.751044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=946d307c3e8044f7950a3a492d47edbd, user_id=3e730ba0339444f5995c0ded7adeee59, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.753942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.768609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1212/2424] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:51 2022] PUT /identity/v3/projects/946d307c3e8044f7950a3a492d47edbd/users/3e730ba0339444f5995c0ded7adeee59/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:50:51.779253 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.779575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.779906 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.806894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] MFA Rules not processed for user `3e730ba0339444f5995c0ded7adeee59`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:51.808134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:51.884302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1213/2425] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:51.889148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:51.951642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb894e0-abf4-47d6-ab75-7631f6f12a0e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:51.956328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:51.957013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:51.957295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:51.957567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:51.958254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:51.960141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:51.989471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1213/2426] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:51.990706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.001948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20144c0-af0b-42cd-8b19-c7be5e640580 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.057584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.059795 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.060079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.060385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.061075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.062988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.082836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1214/2427] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:50:51 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:52.086994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.097061 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c94f3-847c-41fa-914f-98e199b0a981 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.101631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.102277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.102555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.102827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.103544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.105433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.115611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1214/2428] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:52.119547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.130443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7744f-2283-4ddd-bd33-b4b8993a2e62 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.134986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.135848 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.137262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.137544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] PATH_INFO: `/v3/projects/6d570aee9de2496698216ca425d666d8/users/db3dbaab0375476e94d73fd3f7fc8c3f/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.151541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d570aee9de2496698216ca425d666d8, user_id=db3dbaab0375476e94d73fd3f7fc8c3f, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.154791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.166915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1215/2429] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:52 2022] PUT /identity/v3/projects/6d570aee9de2496698216ca425d666d8/users/db3dbaab0375476e94d73fd3f7fc8c3f/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:52.171320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.180910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97557a-11dd-4752-bc68-9efe018e48db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.234495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4a8eb-f2a8-43c7-82fb-8ea2f06d2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.235157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4a8eb-f2a8-43c7-82fb-8ea2f06d2c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.235413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4a8eb-f2a8-43c7-82fb-8ea2f06d2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.235684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4a8eb-f2a8-43c7-82fb-8ea2f06d2c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.236449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4a8eb-f2a8-43c7-82fb-8ea2f06d2c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.238024 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4a8eb-f2a8-43c7-82fb-8ea2f06d2c72 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.247996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1215/2430] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:52.252069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.269216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95aa5c-77b1-4c52-87b2-cb132ac8babd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.274065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.274898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.275162 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.275435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] PATH_INFO: `/v3/projects/6d570aee9de2496698216ca425d666d8/users/db3dbaab0375476e94d73fd3f7fc8c3f/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.277581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d570aee9de2496698216ca425d666d8, user_id=db3dbaab0375476e94d73fd3f7fc8c3f, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.280259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.290182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1216/2431] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:52 2022] PUT /identity/v3/projects/6d570aee9de2496698216ca425d666d8/users/db3dbaab0375476e94d73fd3f7fc8c3f/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:52.301535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.301830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.302113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.329382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] MFA Rules not processed for user `db3dbaab0375476e94d73fd3f7fc8c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:52.330592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:52.398796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1216/2432] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:52.410242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.411943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:52.470794 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eeef88-243f-4b68-a8bb-c488c67cc1c0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.474400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.475040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.475321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.475594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.485350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.486896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.497638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1217/2433] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:52.541498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:52.544081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:52.555787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a02c26-2954-4edf-bd5f-e75111824528 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:52.560516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:52.561134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:52.561414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:52.561693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:52.617056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:52.618480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:52.628388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1217/2434] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:55.597430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:55.598839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:55.657097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff83e2-f205-4553-8ac8-2168059c673b service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:55.661322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:55.662116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:55.662402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:55.662695 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:55.664662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:55.675131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:55.676688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51750c3a-7890-45d5-87e2-669a077b5061 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:55.685767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1218/2435] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:55.921452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:55.992336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8788-964f-448d-87d5-fa7bdd261875 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:55.997614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:55.998478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:55.998758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:55.999027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] PATH_INFO: `/v3/users/3e730ba0339444f5995c0ded7adeee59` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:56.000329 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e730ba0339444f5995c0ded7adeee59)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:56.002272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:56.048116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e730ba0339444f5995c0ded7adeee59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.055275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e730ba0339444f5995c0ded7adeee59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.063999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e730ba0339444f5995c0ded7adeee59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.064646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e730ba0339444f5995c0ded7adeee59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.077005 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e730ba0339444f5995c0ded7adeee59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.083231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1218/2436] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:55 2022] DELETE /identity/v3/users/3e730ba0339444f5995c0ded7adeee59 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:56.087597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856984fe-8128-478d-b37a-64729f33e726 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:56.087887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856984fe-8128-478d-b37a-64729f33e726 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:56.088190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856984fe-8128-478d-b37a-64729f33e726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:56.106283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-856984fe-8128-478d-b37a-64729f33e726 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:56.107475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-856984fe-8128-478d-b37a-64729f33e726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:56.166643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1219/2437] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:56.178173 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:56.179780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:56.258883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c6951-a947-4f53-b8aa-f036ef75f642 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:56.262638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:56.263296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:56.263574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:56.263842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:56.273692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:56.275153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:56.289330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1219/2438] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:50:56.564798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856984fe-8128-478d-b37a-64729f33e726 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:56.566296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-856984fe-8128-478d-b37a-64729f33e726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:56.578337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856984fe-8128-478d-b37a-64729f33e726 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:56.582964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:56.584373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:56.584842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:56.585319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] PATH_INFO: `/v3/projects/946d307c3e8044f7950a3a492d47edbd` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:56.586645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=946d307c3e8044f7950a3a492d47edbd)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:56.588930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:56.635177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946d307c3e8044f7950a3a492d47edbd', 'request_id': 'req-6b81480e-fa2a-461c-b69a-9058f79ba15b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.642512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946d307c3e8044f7950a3a492d47edbd', 'request_id': 'req-6b81480e-fa2a-461c-b69a-9058f79ba15b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.647809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946d307c3e8044f7950a3a492d47edbd', 'request_id': 'req-6b81480e-fa2a-461c-b69a-9058f79ba15b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.662604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] The token cache is being invalidate because project 946d307c3e8044f7950a3a492d47edbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:50:56.662898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.663552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.665121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1220/2439] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:50:56 2022] DELETE /identity/v3/projects/946d307c3e8044f7950a3a492d47edbd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:56.669949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:56.679305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a5a2-17d5-40bd-a3c9-83c79e9c24a9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:56.733632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:56.734404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:56.734687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:56.734962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] PATH_INFO: `/v3/users/db3dbaab0375476e94d73fd3f7fc8c3f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:56.736141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3dbaab0375476e94d73fd3f7fc8c3f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:56.738083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:56.782122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3dbaab0375476e94d73fd3f7fc8c3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.788947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3dbaab0375476e94d73fd3f7fc8c3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.796906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3dbaab0375476e94d73fd3f7fc8c3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.797522 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3dbaab0375476e94d73fd3f7fc8c3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.809245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0cacd51b-4502-4f81-a566-d42f8c0b9002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3dbaab0375476e94d73fd3f7fc8c3f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:56.814999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1220/2440] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:56 2022] DELETE /identity/v3/users/db3dbaab0375476e94d73fd3f7fc8c3f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:57.346930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.348444 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:57.361373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81480e-fa2a-461c-b69a-9058f79ba15b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.418664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.419568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.419962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.420274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] PATH_INFO: `/v3/projects/6d570aee9de2496698216ca425d666d8` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.421453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d570aee9de2496698216ca425d666d8)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.423337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.470112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d570aee9de2496698216ca425d666d8', 'request_id': 'req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:57.478302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d570aee9de2496698216ca425d666d8', 'request_id': 'req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:57.484232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d570aee9de2496698216ca425d666d8', 'request_id': 'req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:57.496059 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] The token cache is being invalidate because project 6d570aee9de2496698216ca425d666d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:50:57.496658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:57.497346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8139dc01-27f4-4439-9a3c-149d0fdc5ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:57.498876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1221/2441] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:50:57 2022] DELETE /identity/v3/projects/6d570aee9de2496698216ca425d666d8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:57.504037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.504565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.504663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.525052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:57.526229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:57.588604 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1221/2442] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:57.597455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.597885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.598303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.618949 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:57.620888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:57.629728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1222/2443] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:57.633846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.644309 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a995478-bb63-41c3-8685-343a3533c574 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.648979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.649629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.649912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.650188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.650915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:50:57.651220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.654214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.666861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1222/2444] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:50:57.672251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.681504 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4689915-25e4-42ea-bb09-46811a8d2213 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.687061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.687713 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.687998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.688315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.689007 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.690886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.715181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1223/2445] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:50:57 2022] POST /identity/v3/projects => generated 389 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:57.719425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.728973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec29922-7ebb-4acd-b5e9-8ae768c2276c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.785207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a497f7-7f56-4c34-ad48-d64924c5c601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.785899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a497f7-7f56-4c34-ad48-d64924c5c601 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.786181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a497f7-7f56-4c34-ad48-d64924c5c601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.786454 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a497f7-7f56-4c34-ad48-d64924c5c601 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.787147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a497f7-7f56-4c34-ad48-d64924c5c601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.789169 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a497f7-7f56-4c34-ad48-d64924c5c601 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.813875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1223/2446] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:50:57 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:57.818058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.827220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.827538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.827822 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.829425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93061-8158-4d75-bf69-a912291fba9c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.834563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.835635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.836632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.836975 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.837785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.839722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.847512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:57.848839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:57.855858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1224/2447] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:57.857394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.869439 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a691d-5856-4bbd-9770-4c985934cfee admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.874980 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.875759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.876114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.876963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.877302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.879461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.896547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1225/2448] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:57.898214 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.910095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cde78-4e7a-44ce-9fa7-b0aa6f1949d2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:57.911277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1224/2449] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:57.916342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e2641-d4eb-4bda-9206-037bb577ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:57.918125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e2641-d4eb-4bda-9206-037bb577ce24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.918481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e2641-d4eb-4bda-9206-037bb577ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.918831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e2641-d4eb-4bda-9206-037bb577ce24 admin admin] PATH_INFO: `/v3/projects/9640fe4cbd1d42b2abc9e77d7507a341/users/dc1a78b15b054153b21e2c4f704b30b4/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.926395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:57.929458 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:57.936546 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e2641-d4eb-4bda-9206-037bb577ce24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9640fe4cbd1d42b2abc9e77d7507a341, user_id=dc1a78b15b054153b21e2c4f704b30b4, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:57.939244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e2641-d4eb-4bda-9206-037bb577ce24 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:57.953824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1226/2450] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:57 2022] PUT /identity/v3/projects/9640fe4cbd1d42b2abc9e77d7507a341/users/dc1a78b15b054153b21e2c4f704b30b4/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:50:57.965313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:57.965818 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:57.966096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:57.994879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] MFA Rules not processed for user `dc1a78b15b054153b21e2c4f704b30b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:57.996418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:58.011037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2709f-c183-49f9-bc7b-363d4fba1598 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.063119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1227/2451] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:50:58.067706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.077511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.078166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.078447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.078722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.132147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b549018-4631-4521-a3d2-09a1e10efe08 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.137755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.138563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.138917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.139266 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.139690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.140084 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.141443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.142161 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.150164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1225/2452] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:50:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:50:58.178402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1228/2453] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:50:58 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:50:58.182180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.192748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65174c-cf53-4725-8424-1c140ec57b41 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.289120 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.289807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.290172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.290471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.291152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.295865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.327323 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1226/2454] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:50:58 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:50:58.331243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.352017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff168c7-05e8-4608-a268-9e041b2fdd85 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.365762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.367294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.367762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.368265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.369209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.371956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.393739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1229/2455] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:58.399036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.414843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc96f-4512-4752-bd41-af2617c5f4bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.434344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.435350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.435703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.436051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] PATH_INFO: `/v3/projects/1d9b1e051b4849c7b9a26dba3e69949d/users/138ab73ca9eb4264833c1e111bd453ea/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.471617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9b1e051b4849c7b9a26dba3e69949d, user_id=138ab73ca9eb4264833c1e111bd453ea, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.474460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.500205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1227/2456] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:58 2022] PUT /identity/v3/projects/1d9b1e051b4849c7b9a26dba3e69949d/users/138ab73ca9eb4264833c1e111bd453ea/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:58.503988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.516418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4f20-4199-4be7-999a-b94ecb1861fe admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.611826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbafdb-6d8d-4236-8961-301bdc40e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.612849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbafdb-6d8d-4236-8961-301bdc40e34c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.613319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbafdb-6d8d-4236-8961-301bdc40e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.613820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbafdb-6d8d-4236-8961-301bdc40e34c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.614736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbafdb-6d8d-4236-8961-301bdc40e34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.616720 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbafdb-6d8d-4236-8961-301bdc40e34c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.631273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1230/2457] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:50:58.633422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.645949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d8918-8c0e-4288-846b-f9a1bdbaff10 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.650948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.652377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.652665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.652943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] PATH_INFO: `/v3/projects/1d9b1e051b4849c7b9a26dba3e69949d/users/138ab73ca9eb4264833c1e111bd453ea/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.655419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9b1e051b4849c7b9a26dba3e69949d, user_id=138ab73ca9eb4264833c1e111bd453ea, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.658975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.673947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1228/2458] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:50:58 2022] PUT /identity/v3/projects/1d9b1e051b4849c7b9a26dba3e69949d/users/138ab73ca9eb4264833c1e111bd453ea/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:50:58.685129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.685934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.686220 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.717788 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] MFA Rules not processed for user `138ab73ca9eb4264833c1e111bd453ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:58.719292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:58.809293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1231/2459] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:50:58.820575 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:58.822572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:58.888709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef964ace-b152-43f4-a07b-d1a4cacb7458 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:58.892787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4db96f-06b6-4341-937a-17eeca3707b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:58.893605 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4db96f-06b6-4341-937a-17eeca3707b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:58.893889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4db96f-06b6-4341-937a-17eeca3707b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:58.894159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4db96f-06b6-4341-937a-17eeca3707b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:58.907247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4db96f-06b6-4341-937a-17eeca3707b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:58.908796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4db96f-06b6-4341-937a-17eeca3707b3 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:58.923357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1229/2460] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:59.068651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:59.131585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db62238-dc06-460c-9202-f80d6d32059c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:59.137016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:59.138002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:59.138287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:59.138565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] PATH_INFO: `/v3/users/dc1a78b15b054153b21e2c4f704b30b4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:59.139851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1a78b15b054153b21e2c4f704b30b4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:59.141767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:59.189819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1a78b15b054153b21e2c4f704b30b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.209163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1a78b15b054153b21e2c4f704b30b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.210202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1a78b15b054153b21e2c4f704b30b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.211163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1a78b15b054153b21e2c4f704b30b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.219773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1a78b15b054153b21e2c4f704b30b4'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.233163 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1232/2461] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:59 2022] DELETE /identity/v3/users/dc1a78b15b054153b21e2c4f704b30b4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:59.237828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:59.238148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:59.238425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:59.264723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:50:59.264723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:59.329668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1230/2462] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:50:59.340447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:59.341865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:59.404038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473cbec-6134-4c9d-b233-6eb8f6c1e9cd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:59.407692 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:59.408367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:59.408651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:59.408924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:59.417777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:59.419185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:59.428710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1233/2463] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:50:59.787937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:59.789750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:50:59.803112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac796-a324-48b3-ac44-73a6f7a22c11 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:50:59.819097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:50:59.820912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:50:59.821372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:50:59.821812 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] PATH_INFO: `/v3/projects/9640fe4cbd1d42b2abc9e77d7507a341` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:50:59.824156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9640fe4cbd1d42b2abc9e77d7507a341)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:50:59.827023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:50:59.887981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9640fe4cbd1d42b2abc9e77d7507a341', 'request_id': 'req-c42a4c50-a270-475c-b3c0-dfeb077f8915'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.898845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9640fe4cbd1d42b2abc9e77d7507a341', 'request_id': 'req-c42a4c50-a270-475c-b3c0-dfeb077f8915'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.908392 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9640fe4cbd1d42b2abc9e77d7507a341', 'request_id': 'req-c42a4c50-a270-475c-b3c0-dfeb077f8915'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.927602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] The token cache is being invalidate because project 9640fe4cbd1d42b2abc9e77d7507a341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:50:59.928235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.928996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:50:59.930993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1231/2464] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:50:59 2022] DELETE /identity/v3/projects/9640fe4cbd1d42b2abc9e77d7507a341 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:50:59.935474 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:50:59.945296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf121d-68f6-4b67-a757-07635a1d0f6e service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:00.013025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:00.013910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.014194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.014481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] PATH_INFO: `/v3/users/138ab73ca9eb4264833c1e111bd453ea` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.015710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138ab73ca9eb4264833c1e111bd453ea)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:00.017650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:00.062974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138ab73ca9eb4264833c1e111bd453ea'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.077150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138ab73ca9eb4264833c1e111bd453ea'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.078323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138ab73ca9eb4264833c1e111bd453ea'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.079210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138ab73ca9eb4264833c1e111bd453ea'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.088693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f26e2234-c68d-4193-b726-67164630eef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138ab73ca9eb4264833c1e111bd453ea'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.107193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1234/2465] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:50:59 2022] DELETE /identity/v3/users/138ab73ca9eb4264833c1e111bd453ea => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:00.425538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:00.426957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:00.443419 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4c50-a270-475c-b3c0-dfeb077f8915 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:00.508741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:00.509612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.509962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.510313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] PATH_INFO: `/v3/projects/1d9b1e051b4849c7b9a26dba3e69949d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.512428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9b1e051b4849c7b9a26dba3e69949d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:00.514478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:00.571650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9b1e051b4849c7b9a26dba3e69949d', 'request_id': 'req-e91af795-37fc-43df-a2c4-80e026de1610'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.588140 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9b1e051b4849c7b9a26dba3e69949d', 'request_id': 'req-e91af795-37fc-43df-a2c4-80e026de1610'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.594397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9b1e051b4849c7b9a26dba3e69949d', 'request_id': 'req-e91af795-37fc-43df-a2c4-80e026de1610'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.608739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] The token cache is being invalidate because project 1d9b1e051b4849c7b9a26dba3e69949d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:00.609154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.609878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e91af795-37fc-43df-a2c4-80e026de1610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:00.616086 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1232/2466] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:00 2022] DELETE /identity/v3/projects/1d9b1e051b4849c7b9a26dba3e69949d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:00.622786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.623192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.623547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.643169 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:00.644505 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:00.718277 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1235/2467] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:00.725973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.726402 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.726750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.757161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:00.758665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:00.770562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1233/2468] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:00.771920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:00.781701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa8992-9e53-4a48-b6a8-7198cef8a764 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:00.797902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:00.798658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.798968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.799228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.800404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:00.800404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:00.801965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:00.830555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1236/2469] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:51:00.834586 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:00.844456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcaaec5-a40c-4deb-94c9-4371dab8fe65 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:00.849099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:00.849777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.850056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.850327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.851098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:00.853031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:00.879802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1234/2470] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:00 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:00.883876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:00.894948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84233658-7c3d-4b15-b303-938e1c798f94 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:00.960757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:00.961467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:00.961719 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:00.961993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:00.962684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:00.965244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:00.984323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1237/2471] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:00 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:00.988485 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:00.999109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e540a-ee90-4bcd-ad25-b34e4fd52170 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:01.004230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:01.004887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:01.005168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:01.005442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:01.006171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:01.008052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:01.019659 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1235/2472] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:01.023676 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:01.033892 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a1c2a-d21c-4231-9cee-391d3a0656d6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:01.040646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56bd2b-1307-4c16-87f3-3bfe6072d4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:01.041464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56bd2b-1307-4c16-87f3-3bfe6072d4db admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:01.041863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56bd2b-1307-4c16-87f3-3bfe6072d4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:01.042367 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56bd2b-1307-4c16-87f3-3bfe6072d4db admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:01.043206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56bd2b-1307-4c16-87f3-3bfe6072d4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:01.045304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56bd2b-1307-4c16-87f3-3bfe6072d4db admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:01.059925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1238/2473] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:01.061440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:01.074581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f3419-5431-4e95-9050-8ca2e9b5ce2a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:01.079465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:01.080379 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:01.080662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:01.080934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] PATH_INFO: `/v3/projects/0a3303ba498d4f7292dba7bf382893ea/users/80327a16ae754847b3e73d9f3e9b9dfe/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:01.096210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3303ba498d4f7292dba7bf382893ea, user_id=80327a16ae754847b3e73d9f3e9b9dfe, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:01.098846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:01.111624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1236/2474] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:01 2022] PUT /identity/v3/projects/0a3303ba498d4f7292dba7bf382893ea/users/80327a16ae754847b3e73d9f3e9b9dfe/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:01.123137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:01.123472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:01.123749 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:01.153850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] MFA Rules not processed for user `80327a16ae754847b3e73d9f3e9b9dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:01.155079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:01.219394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1239/2475] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:01.230641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:01.232399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:01.296108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0409b8-382f-4066-b870-80f4485978e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:01.299669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f1e9f-a246-49b5-b025-4bb5b5675c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:01.300351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f1e9f-a246-49b5-b025-4bb5b5675c3e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:01.300632 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f1e9f-a246-49b5-b025-4bb5b5675c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:01.300905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f1e9f-a246-49b5-b025-4bb5b5675c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:01.311393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f1e9f-a246-49b5-b025-4bb5b5675c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:01.312969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f1e9f-a246-49b5-b025-4bb5b5675c3e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:01.324656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1237/2476] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:09.798030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:09.857616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c309f-7b59-4df5-914a-a87697e92ad5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:09.862021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:09.862861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:09.863144 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:09.863419 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] PATH_INFO: `/v3/users/80327a16ae754847b3e73d9f3e9b9dfe` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:09.864612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] RBAC: Authorizing `identity:delete_user(user_id=80327a16ae754847b3e73d9f3e9b9dfe)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:09.866502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:09.909454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80327a16ae754847b3e73d9f3e9b9dfe'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:09.921802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80327a16ae754847b3e73d9f3e9b9dfe'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:09.922453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80327a16ae754847b3e73d9f3e9b9dfe'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:09.923064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80327a16ae754847b3e73d9f3e9b9dfe'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:09.930303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80327a16ae754847b3e73d9f3e9b9dfe'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:09.943335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1240/2477] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:09 2022] DELETE /identity/v3/users/80327a16ae754847b3e73d9f3e9b9dfe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:09.947961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:09.948715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:09.949003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:09.965967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:09.967147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:10.022340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1238/2478] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:10.032762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.034168 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:10.087389 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313b4ab-75ba-4880-8c99-9657f2ad286c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.091427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dfdbc-0cab-473e-8f9e-3b186ccf53c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.092057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dfdbc-0cab-473e-8f9e-3b186ccf53c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.092361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dfdbc-0cab-473e-8f9e-3b186ccf53c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.092635 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dfdbc-0cab-473e-8f9e-3b186ccf53c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.106035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dfdbc-0cab-473e-8f9e-3b186ccf53c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.107600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dfdbc-0cab-473e-8f9e-3b186ccf53c1 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.117486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1241/2479] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:10.375697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.377155 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:10.391003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e039a42-308d-434a-9b65-7e9926deb2eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.396095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.397118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.397464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.397808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] PATH_INFO: `/v3/projects/0a3303ba498d4f7292dba7bf382893ea` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.399122 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3303ba498d4f7292dba7bf382893ea)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.401141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.451584 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3303ba498d4f7292dba7bf382893ea', 'request_id': 'req-8bb21516-8900-46a1-8748-3711afe958e7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:10.457423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3303ba498d4f7292dba7bf382893ea', 'request_id': 'req-8bb21516-8900-46a1-8748-3711afe958e7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:10.462596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3303ba498d4f7292dba7bf382893ea', 'request_id': 'req-8bb21516-8900-46a1-8748-3711afe958e7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:10.475518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] The token cache is being invalidate because project 0a3303ba498d4f7292dba7bf382893ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:10.475858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:10.476609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8bb21516-8900-46a1-8748-3711afe958e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:10.478161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1239/2480] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:10 2022] DELETE /identity/v3/projects/0a3303ba498d4f7292dba7bf382893ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:10.483158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.483447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.483723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.501225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:10.502413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:10.561860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1242/2481] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:10.569803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.570089 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.570359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.590038 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:10.591439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:10.600949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1240/2482] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:10.605574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.615291 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5b985-645e-4964-bbe2-c25b21a8407a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.621230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.621842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.622124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.622402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.623133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:10.623440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.625053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.637363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1243/2483] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:51:10.641908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.650909 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011021cc-2194-417f-901f-12b3cfd05815 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.655696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.656376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.656657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.656929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.657611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.659534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.694307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1241/2484] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:10 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:10.700417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.710563 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01428bbd-d69b-44be-90e5-9f13ef471b19 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.765643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.766303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.766582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.766853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.767541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.769464 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.788985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1244/2485] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:10 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:10.793716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.803611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6f70-901e-46c9-8bcd-82c62379bbcc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.808305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.808951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.809236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.809507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.810227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.811800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.821765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1242/2486] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:10.825579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.835227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2e53c-0470-431a-8351-9bc8f738335c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.840448 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047ccc2-7905-451e-8699-038f92fdcebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.841100 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047ccc2-7905-451e-8699-038f92fdcebe admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.841377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047ccc2-7905-451e-8699-038f92fdcebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.841671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047ccc2-7905-451e-8699-038f92fdcebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.842415 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047ccc2-7905-451e-8699-038f92fdcebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.843982 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047ccc2-7905-451e-8699-038f92fdcebe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.854029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1245/2487] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:10.858794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:10.868468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dd3c9-5d41-49bc-99f2-c96608460bd2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:10.873388 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:10.874191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.874475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.874748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] PATH_INFO: `/v3/projects/02c58b48957e4aa4b0a1d795acbca76f/users/391f714f4cf24949b8c7709b310f4122/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.890229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c58b48957e4aa4b0a1d795acbca76f, user_id=391f714f4cf24949b8c7709b310f4122, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:10.892902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:10.905746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1243/2488] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:10 2022] PUT /identity/v3/projects/02c58b48957e4aa4b0a1d795acbca76f/users/391f714f4cf24949b8c7709b310f4122/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:10.915945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:10.916289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:10.916575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:10.943799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] MFA Rules not processed for user `391f714f4cf24949b8c7709b310f4122`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:10.945051 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:11.007856 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1246/2489] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:11.012368 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.072057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8cb56-1c23-49a8-8420-f598133d5b37 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.076835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.077484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.077759 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.078023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.078711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.080649 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.106291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1244/2490] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:11 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:11.110340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.121903 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aab124-0877-4261-b581-9129ae1ed2fb None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.175753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.176435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.176716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.176989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.177677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.179582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.199129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1247/2491] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:11 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:11.202998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.212962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29554872-9c74-42c6-99cb-41dc20ebe741 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.218129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.218774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.219058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.219334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.220054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.221936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.232473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1245/2492] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:11.236275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.245922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6df1-568d-4754-a6cf-a05d707563c4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.251623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b259-7e93-40ad-8466-946489c13ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.252302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b259-7e93-40ad-8466-946489c13ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.252582 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b259-7e93-40ad-8466-946489c13ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.253140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b259-7e93-40ad-8466-946489c13ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.253859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b259-7e93-40ad-8466-946489c13ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.255457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b259-7e93-40ad-8466-946489c13ef3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.265665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1248/2493] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:51:11.269971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.280979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bb752-029d-44ca-ba8f-ae5d6b88f8b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.285985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.286914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.287189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.287453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] PATH_INFO: `/v3/projects/f9d303aea88e49f4a68de188a76e839d/users/aad5c63522ac4588bfe8003fa44bef59/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.302346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d303aea88e49f4a68de188a76e839d, user_id=aad5c63522ac4588bfe8003fa44bef59, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.305011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.318010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1246/2494] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:11 2022] PUT /identity/v3/projects/f9d303aea88e49f4a68de188a76e839d/users/aad5c63522ac4588bfe8003fa44bef59/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:11.330594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.330901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.331184 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.358530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] MFA Rules not processed for user `aad5c63522ac4588bfe8003fa44bef59`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:11.359746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:11.426990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1249/2495] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:11.431420 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.490833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6767-1684-4e9a-b872-3395d235a196 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.496152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.496826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.497133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.497409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.498098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.500353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.526175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1247/2496] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:11 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:11.530599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.540776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165e72a-20e7-4998-8694-6f31dd6c34a0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.596137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.596830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.597112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.597385 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.598067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.599974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.619904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1250/2497] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:11 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:11.624436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.634550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d49651-726e-49ee-b302-3835da7fcd97 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.639729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.640396 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.640678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.640953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.641668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.643236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.653548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1248/2498] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:11.657437 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.666974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c595ab2-e358-48a7-ac1a-d119785f9e11 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.671554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.672433 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.672709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.672979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] PATH_INFO: `/v3/projects/4e938a54acee402895841040c0904db7/users/1fdefb64f178460da3607fc0c93a4535/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.687998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e938a54acee402895841040c0904db7, user_id=1fdefb64f178460da3607fc0c93a4535, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.690587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.704707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1251/2499] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:11 2022] PUT /identity/v3/projects/4e938a54acee402895841040c0904db7/users/1fdefb64f178460da3607fc0c93a4535/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:11.708807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.719009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134a6cd-d397-4432-a3e5-b5786dfde38b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.779467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c524c-8d3e-4b29-b778-8faa7fe1f521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.780134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c524c-8d3e-4b29-b778-8faa7fe1f521 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.780469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c524c-8d3e-4b29-b778-8faa7fe1f521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.780747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c524c-8d3e-4b29-b778-8faa7fe1f521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.781701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c524c-8d3e-4b29-b778-8faa7fe1f521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.783033 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c524c-8d3e-4b29-b778-8faa7fe1f521 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.793741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1249/2500] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:11.797323 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.808028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a3097-974e-4cc7-9e1e-068611d7a8f6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:11.813047 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:11.813946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.814235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.814513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] PATH_INFO: `/v3/projects/4e938a54acee402895841040c0904db7/users/1fdefb64f178460da3607fc0c93a4535/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.816528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e938a54acee402895841040c0904db7, user_id=1fdefb64f178460da3607fc0c93a4535, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:11.819158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:11.830283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1252/2501] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:11 2022] PUT /identity/v3/projects/4e938a54acee402895841040c0904db7/users/1fdefb64f178460da3607fc0c93a4535/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:11.840447 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:11.840739 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:11.841013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:11.866564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] MFA Rules not processed for user `1fdefb64f178460da3607fc0c93a4535`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:11.867748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:11.935278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1250/2502] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:11.946514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:11.948332 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:12.003311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59480-ec5f-4035-9f1f-1039e7736a0a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:12.006907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3e283-8c40-4968-b516-2dc2f0a7111b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:12.007531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3e283-8c40-4968-b516-2dc2f0a7111b service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:12.007810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3e283-8c40-4968-b516-2dc2f0a7111b service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:12.008082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3e283-8c40-4968-b516-2dc2f0a7111b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:12.063616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3e283-8c40-4968-b516-2dc2f0a7111b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:12.065094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3e283-8c40-4968-b516-2dc2f0a7111b service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:12.074431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1253/2503] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:13.536681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:13.546555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e91fa-f685-4e59-9d5d-8e1e61cdfaba None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:13.550078 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:13.550703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:13.550983 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:13.551256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:13.604850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:13.606261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:13.615745 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1251/2504] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:14.054222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:14.054694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:14.054931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:14.073962 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:14.075262 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:14.128619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:14.134137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1254/2505] 10.222.0.73 () {58 vars in 967 bytes} [Thu May 12 17:51:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:14.137757 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f29c39-d975-486a-94b1-9d891e8f79bd service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:14.141527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:14.142514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:14.142514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:14.143192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:14.147776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:14.149128 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:14.153754 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:14.155369 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:14.165036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1252/2506] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:14.220630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a8ce4-b9df-4b37-b6a7-f9671165bebc None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:14.226321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf35d16-7af0-4137-805f-3642031f3b81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:14.226976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf35d16-7af0-4137-805f-3642031f3b81 service swift] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:14.227260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf35d16-7af0-4137-805f-3642031f3b81 service swift] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:14.227535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf35d16-7af0-4137-805f-3642031f3b81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:14.236034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf35d16-7af0-4137-805f-3642031f3b81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:14.237652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf35d16-7af0-4137-805f-3642031f3b81 service swift] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:14.248118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1255/2507] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:51:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 12 17:51:17.988389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:18.047779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3282-fddc-4837-b519-330074e82874 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:18.052219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:18.053073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:18.053351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:18.053619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] PATH_INFO: `/v3/users/391f714f4cf24949b8c7709b310f4122` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:18.054774 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=391f714f4cf24949b8c7709b310f4122)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:18.057074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:18.104123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391f714f4cf24949b8c7709b310f4122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.111372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391f714f4cf24949b8c7709b310f4122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.118533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391f714f4cf24949b8c7709b310f4122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.119139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391f714f4cf24949b8c7709b310f4122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.131546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391f714f4cf24949b8c7709b310f4122'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.137475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1253/2508] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:17 2022] DELETE /identity/v3/users/391f714f4cf24949b8c7709b310f4122 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:18.141771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:18.142234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:18.142347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:18.159773 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:18.161061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:18.217845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1256/2509] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:18.227133 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:18.228539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:18.284482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86c130-0982-4214-a359-144de77a6d7a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:18.288067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:18.288726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:18.289004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:18.289275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:18.298439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:18.299850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:18.309108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1254/2510] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:18.565829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:18.567273 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:18.580723 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e48af-2265-48c6-9b02-f3b01f5920b8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:18.585584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:18.587351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:18.587633 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:18.587904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] PATH_INFO: `/v3/projects/02c58b48957e4aa4b0a1d795acbca76f` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:18.589820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c58b48957e4aa4b0a1d795acbca76f)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:18.591761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:18.638855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c58b48957e4aa4b0a1d795acbca76f', 'request_id': 'req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.645955 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c58b48957e4aa4b0a1d795acbca76f', 'request_id': 'req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.651228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c58b48957e4aa4b0a1d795acbca76f', 'request_id': 'req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.662518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] The token cache is being invalidate because project 02c58b48957e4aa4b0a1d795acbca76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:18.662804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.663427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.665013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1257/2511] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:18 2022] DELETE /identity/v3/projects/02c58b48957e4aa4b0a1d795acbca76f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:18.668953 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:18.678762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd606540-d2f5-4aad-ab38-ab48e2dc1e87 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:18.733791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:18.734642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:18.734922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:18.735193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] PATH_INFO: `/v3/users/aad5c63522ac4588bfe8003fa44bef59` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:18.736393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad5c63522ac4588bfe8003fa44bef59)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:18.738687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:18.784797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad5c63522ac4588bfe8003fa44bef59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.791148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad5c63522ac4588bfe8003fa44bef59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.799117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad5c63522ac4588bfe8003fa44bef59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.799787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad5c63522ac4588bfe8003fa44bef59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.811210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad5c63522ac4588bfe8003fa44bef59'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:18.817218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1255/2512] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:18 2022] DELETE /identity/v3/users/aad5c63522ac4588bfe8003fa44bef59 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:19.095972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:19.097504 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:19.113948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da68c97-06f5-42bc-aafc-eb9ddff3cf8b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:19.171194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:19.172069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:19.172461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:19.172736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] PATH_INFO: `/v3/projects/f9d303aea88e49f4a68de188a76e839d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:19.173917 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d303aea88e49f4a68de188a76e839d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:19.175809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:19.219533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d303aea88e49f4a68de188a76e839d', 'request_id': 'req-5641b122-ecf0-48a2-8302-049c48ec0c2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.226571 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d303aea88e49f4a68de188a76e839d', 'request_id': 'req-5641b122-ecf0-48a2-8302-049c48ec0c2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.232830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d303aea88e49f4a68de188a76e839d', 'request_id': 'req-5641b122-ecf0-48a2-8302-049c48ec0c2a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.243243 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] The token cache is being invalidate because project f9d303aea88e49f4a68de188a76e839d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:19.243525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.244153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.246210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1258/2513] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:19 2022] DELETE /identity/v3/projects/f9d303aea88e49f4a68de188a76e839d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:19.250107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:19.251519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:19.265111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4e7d-e4bd-481d-a934-b8d0e3d6c44c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:19.318593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:19.319359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:19.319640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:19.319914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] PATH_INFO: `/v3/users/1fdefb64f178460da3607fc0c93a4535` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:19.321299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdefb64f178460da3607fc0c93a4535)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:19.323854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:19.366603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdefb64f178460da3607fc0c93a4535'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.373800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdefb64f178460da3607fc0c93a4535'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.381035 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdefb64f178460da3607fc0c93a4535'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.381646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdefb64f178460da3607fc0c93a4535'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.393240 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e13ea420-6d28-480e-83b4-1efb70741a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdefb64f178460da3607fc0c93a4535'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.398884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1256/2514] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:19 2022] DELETE /identity/v3/users/1fdefb64f178460da3607fc0c93a4535 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:19.822839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:19.825297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:19.848156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641b122-ecf0-48a2-8302-049c48ec0c2a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:19.903851 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:19.904741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:19.905016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:19.905285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] PATH_INFO: `/v3/projects/4e938a54acee402895841040c0904db7` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:19.906463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e938a54acee402895841040c0904db7)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:19.908524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:19.958512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e938a54acee402895841040c0904db7', 'request_id': 'req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.966414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e938a54acee402895841040c0904db7', 'request_id': 'req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.972841 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e938a54acee402895841040c0904db7', 'request_id': 'req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.986552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] The token cache is being invalidate because project 4e938a54acee402895841040c0904db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:19.986832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.987485 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-32e1f5c1-42f3-4a23-a6ce-e17433ee5209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:19.991454 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1259/2515] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:19 2022] DELETE /identity/v3/projects/4e938a54acee402895841040c0904db7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:19.994201 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:19.994487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:19.994762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.014395 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:20.016455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:20.085997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1257/2516] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:20.094174 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.094469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.094742 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.120751 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:20.122079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:20.130368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1260/2517] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:20.134639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.145374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663857-0d67-4ef0-adce-85679e643b89 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.149916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.150571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.150848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.151727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.152576 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:20.152912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.154619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.166701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1258/2518] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:51:20.171029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.185119 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c23e5-70c0-4760-9459-aae29d293d68 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.190756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.191406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.191684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.192077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.192756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.194762 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.221484 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1261/2519] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:20 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:20.225480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.235771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9370c75-2a80-42a9-bb77-f4d644774412 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.290324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.290993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.291254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.291525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.292284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.294384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.317805 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1259/2520] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:20 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:20.321679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.332645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e64b3-7bbf-42a3-a3de-65cfc7957cb2 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.341902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.342846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.343322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.343777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.344710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.347077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.363573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1262/2521] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:20.364492 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.376040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f40433-5ebf-4d09-b8c3-6fc8a49a81a7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.382202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986a6b6-7f9a-4ae5-a13a-622127cce911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.383013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986a6b6-7f9a-4ae5-a13a-622127cce911 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.383360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986a6b6-7f9a-4ae5-a13a-622127cce911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.383702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986a6b6-7f9a-4ae5-a13a-622127cce911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.384529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986a6b6-7f9a-4ae5-a13a-622127cce911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.387189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986a6b6-7f9a-4ae5-a13a-622127cce911 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.400913 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1260/2522] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:20.405698 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.419232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca72fbe-5d5d-49ab-8930-5a1e71fccd73 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.431275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.432278 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.432858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.433374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] PATH_INFO: `/v3/projects/53e97d53681a48b8a7a3de276d87f85d/users/bf1b31c3bf144ef0bb3a932b14eeab17/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.449816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e97d53681a48b8a7a3de276d87f85d, user_id=bf1b31c3bf144ef0bb3a932b14eeab17, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.452680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.478463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1263/2523] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:20 2022] PUT /identity/v3/projects/53e97d53681a48b8a7a3de276d87f85d/users/bf1b31c3bf144ef0bb3a932b14eeab17/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:20.480892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.481374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.481795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.517014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] MFA Rules not processed for user `bf1b31c3bf144ef0bb3a932b14eeab17`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:20.518669 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:20.610183 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1261/2524] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:20.619471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:20.621935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:20.700652 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd4874-369c-4d3c-95cf-5607e6a24a92 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:20.704307 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:20.704945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:20.705403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:20.705659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:20.714129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:20.715663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:20.724660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1264/2525] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:25.475244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:25.476832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:25.533463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a66a-1f74-4176-8d82-c8e363b6f05e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:25.537021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:25.537664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:25.537959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:25.538219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:25.539982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:25.603417 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:25.605071 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:25.615236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1262/2526] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:51:29.022521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.024430 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:29.040305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765a647-b211-4bee-bc18-8cfbcccacb69 service glance] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.046366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.047328 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.047620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.047899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] PATH_INFO: `/v3/users/1cc16878bc6a445f8a66dfcdc54a7e7b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.049234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc16878bc6a445f8a66dfcdc54a7e7b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.051180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.102002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc16878bc6a445f8a66dfcdc54a7e7b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.114570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc16878bc6a445f8a66dfcdc54a7e7b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.115248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc16878bc6a445f8a66dfcdc54a7e7b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.115845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc16878bc6a445f8a66dfcdc54a7e7b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.122619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-28544bb4-d3ee-466f-bcf9-76416d0f8d09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc16878bc6a445f8a66dfcdc54a7e7b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.137217 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1265/2527] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:29 2022] DELETE /identity/v3/users/1cc16878bc6a445f8a66dfcdc54a7e7b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:29.310390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.327303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71576f0-1e40-41aa-89f1-54a3eb4c944e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.388142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.389041 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.389521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.389808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] PATH_INFO: `/v3/projects/0eee0523f68a4d8dbc6db8de3a3f6b04` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.391104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eee0523f68a4d8dbc6db8de3a3f6b04)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.393091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.440002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eee0523f68a4d8dbc6db8de3a3f6b04', 'request_id': 'req-a3c0e302-5016-4437-a0d8-1c2aa0410460'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.446882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eee0523f68a4d8dbc6db8de3a3f6b04', 'request_id': 'req-a3c0e302-5016-4437-a0d8-1c2aa0410460'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.452911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eee0523f68a4d8dbc6db8de3a3f6b04', 'request_id': 'req-a3c0e302-5016-4437-a0d8-1c2aa0410460'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.465243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] The token cache is being invalidate because project 0eee0523f68a4d8dbc6db8de3a3f6b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:29.465527 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.466144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a3c0e302-5016-4437-a0d8-1c2aa0410460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:29.467668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1263/2528] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:29 2022] DELETE /identity/v3/projects/0eee0523f68a4d8dbc6db8de3a3f6b04 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:29.472847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.473141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.473426 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.492263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:29.493459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:29.554258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1266/2529] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:29.562809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.563100 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.563370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.581908 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:29.583104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:29.592344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1264/2530] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:29.596453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.605987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bd1dd-4b1f-4536-a615-a99dcd8d3722 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.612524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.612524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.612524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.612524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.612989 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:29.613092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.614657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.628286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1267/2531] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:51:29.631734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.641798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dd848-16e1-4cc3-bc2b-1c7c37075f37 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.646476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.647127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.647405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.647674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.648433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.650316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.679374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1265/2532] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:29 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:29.684810 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.694802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac90bdb-ace0-4644-87c9-b0379d53f7fa admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.754269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.754943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.755228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.755511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.756225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.758131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.779948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1268/2533] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:29 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:29.783374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.793625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951e39-481e-4d19-934f-b3e41cf646fc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.798295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.798966 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.799245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.799516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.800278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.801858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.812810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1266/2534] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:29.816400 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.827611 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11daa-3fa9-4b26-9ffc-656ecf8f220b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.832380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9032e-7f85-46ee-8818-2b95026b8e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.833045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9032e-7f85-46ee-8818-2b95026b8e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.833333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9032e-7f85-46ee-8818-2b95026b8e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.833612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9032e-7f85-46ee-8818-2b95026b8e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.834345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9032e-7f85-46ee-8818-2b95026b8e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.835928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9032e-7f85-46ee-8818-2b95026b8e82 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.846569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1269/2535] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:29.850903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:29.861579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e3cf9-059e-486e-b1cb-fd51a4c99bab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:29.866104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:29.866962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.867238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.867508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] PATH_INFO: `/v3/projects/ad8d85059f904125b7bc5ec8fda67332/users/ef363dda5c254bdd94a884f4cd253d0e/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.883976 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8d85059f904125b7bc5ec8fda67332, user_id=ef363dda5c254bdd94a884f4cd253d0e, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:29.887416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:29.899848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1267/2536] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:29 2022] PUT /identity/v3/projects/ad8d85059f904125b7bc5ec8fda67332/users/ef363dda5c254bdd94a884f4cd253d0e/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:29.911005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:29.911302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:29.911585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:29.941672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] MFA Rules not processed for user `ef363dda5c254bdd94a884f4cd253d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:29.942888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:30.006561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1270/2537] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:30.011386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.072731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cea7b-b304-4d10-8bf7-4deaf25a7e41 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.077310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.077959 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.078238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.078508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.079225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.081147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.106912 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1268/2538] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:30 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:30.110965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.123365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df164ca-932e-4b3c-83cc-de7597b7cc03 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.179017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.179685 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.179976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.180317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.181025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.183513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.202600 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1271/2539] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:30 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:30.207221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.218711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed739f-c383-434e-84e3-b8d212a86d06 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.223553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.224247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.224532 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.224811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.225540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.227126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.238316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1269/2540] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:30.243279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.253995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9777b-7fb4-44fd-8774-e793ee3fecf6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.258733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.259684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.259964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.260325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] PATH_INFO: `/v3/projects/ff3d25b727384ed7833553188a08d188/users/927481a9afcb49b88e5de2d950cbbe8c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.275204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3d25b727384ed7833553188a08d188, user_id=927481a9afcb49b88e5de2d950cbbe8c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.278650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.293248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1272/2541] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:30 2022] PUT /identity/v3/projects/ff3d25b727384ed7833553188a08d188/users/927481a9afcb49b88e5de2d950cbbe8c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:30.297306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.307112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e35b4-79a7-4638-9514-8b62157db988 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.365337 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1543506-d5e8-485d-afd7-90b8def01c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.366028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1543506-d5e8-485d-afd7-90b8def01c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.366199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1543506-d5e8-485d-afd7-90b8def01c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.366487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1543506-d5e8-485d-afd7-90b8def01c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.367188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1543506-d5e8-485d-afd7-90b8def01c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.368809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1543506-d5e8-485d-afd7-90b8def01c60 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.379146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1270/2542] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:30.383515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.394046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6666d94-2042-4672-a1f7-ae3025ff6e52 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.398578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.399491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.399772 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.400073 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] PATH_INFO: `/v3/projects/ff3d25b727384ed7833553188a08d188/users/927481a9afcb49b88e5de2d950cbbe8c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.402844 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3d25b727384ed7833553188a08d188, user_id=927481a9afcb49b88e5de2d950cbbe8c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.405607 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.415966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1273/2543] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:30 2022] PUT /identity/v3/projects/ff3d25b727384ed7833553188a08d188/users/927481a9afcb49b88e5de2d950cbbe8c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:30.427638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.427926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.428235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.454876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] MFA Rules not processed for user `927481a9afcb49b88e5de2d950cbbe8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:30.456193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:30.523831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1271/2544] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:30.536335 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:30.538079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:30.598295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095dfe-95b9-4e77-829c-56682cccd289 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:30.601996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b924f5-ed87-450f-b721-739e767f22a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:30.602643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b924f5-ed87-450f-b721-739e767f22a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:30.602925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b924f5-ed87-450f-b721-739e767f22a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:30.603201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b924f5-ed87-450f-b721-739e767f22a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:30.613413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b924f5-ed87-450f-b721-739e767f22a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:30.614865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b924f5-ed87-450f-b721-739e767f22a8 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:30.625091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1274/2545] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:31.003372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:31.066596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4327e-07ab-46e4-87c9-73d1678eedda None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:31.071230 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:31.072243 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:31.072596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:31.074057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] PATH_INFO: `/v3/users/ef363dda5c254bdd94a884f4cd253d0e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:31.075256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef363dda5c254bdd94a884f4cd253d0e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:31.077220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:31.123951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef363dda5c254bdd94a884f4cd253d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.131142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef363dda5c254bdd94a884f4cd253d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.139831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef363dda5c254bdd94a884f4cd253d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.140478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef363dda5c254bdd94a884f4cd253d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.152027 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef363dda5c254bdd94a884f4cd253d0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.157671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1272/2546] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:31 2022] DELETE /identity/v3/users/ef363dda5c254bdd94a884f4cd253d0e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:31.162173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:31.162634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:31.162758 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:31.179465 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:31.180700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:31.239208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1275/2547] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:31.249948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:31.251322 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:31.308542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84927be4-915a-429e-84b0-d03aac795c88 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:31.312085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:31.312748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:31.313022 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:31.313291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:31.321897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:31.323896 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:31.333888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1273/2548] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:31.599741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:31.601263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:31.613556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ee20c-acfa-49c5-801b-68d43bd7ca12 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:31.624046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:31.624942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:31.625221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:31.625515 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] PATH_INFO: `/v3/projects/ad8d85059f904125b7bc5ec8fda67332` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:31.626688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8d85059f904125b7bc5ec8fda67332)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:31.628874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:31.677504 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8d85059f904125b7bc5ec8fda67332', 'request_id': 'req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.687056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8d85059f904125b7bc5ec8fda67332', 'request_id': 'req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.693083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8d85059f904125b7bc5ec8fda67332', 'request_id': 'req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.704001 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] The token cache is being invalidate because project ad8d85059f904125b7bc5ec8fda67332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:31.704313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.704942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.706456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1276/2549] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:31 2022] DELETE /identity/v3/projects/ad8d85059f904125b7bc5ec8fda67332 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:31.710415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:31.720980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4bbf5-d628-4d8d-a72d-6ce9baa89e5a service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:31.778514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:31.779401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:31.779677 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:31.779947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] PATH_INFO: `/v3/users/927481a9afcb49b88e5de2d950cbbe8c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:31.781262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=927481a9afcb49b88e5de2d950cbbe8c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:31.783181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:31.826570 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927481a9afcb49b88e5de2d950cbbe8c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.833338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927481a9afcb49b88e5de2d950cbbe8c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.841925 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927481a9afcb49b88e5de2d950cbbe8c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.842554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927481a9afcb49b88e5de2d950cbbe8c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.854911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddcef9b6-12c9-4dc5-b0da-d2554827d7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927481a9afcb49b88e5de2d950cbbe8c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:31.860841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1274/2550] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:31 2022] DELETE /identity/v3/users/927481a9afcb49b88e5de2d950cbbe8c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:32.129677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.131127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:32.145901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64cb92-9acb-46a6-8fb1-ef9f71e6289f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.201477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.202337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.202622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.202897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] PATH_INFO: `/v3/projects/ff3d25b727384ed7833553188a08d188` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.204063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3d25b727384ed7833553188a08d188)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.205987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.252538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3d25b727384ed7833553188a08d188', 'request_id': 'req-7a5ff6b3-04bd-4788-856c-774531e02fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:32.261132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3d25b727384ed7833553188a08d188', 'request_id': 'req-7a5ff6b3-04bd-4788-856c-774531e02fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:32.266919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3d25b727384ed7833553188a08d188', 'request_id': 'req-7a5ff6b3-04bd-4788-856c-774531e02fc9'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:32.279403 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] The token cache is being invalidate because project ff3d25b727384ed7833553188a08d188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:32.279693 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:32.280357 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-7a5ff6b3-04bd-4788-856c-774531e02fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:32.281885 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1277/2551] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:32 2022] DELETE /identity/v3/projects/ff3d25b727384ed7833553188a08d188 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:32.286704 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.286992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.287286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.304969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:32.306166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:32.365600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1275/2552] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:32.373621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.373912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.374194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.392527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:32.393715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:32.401388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1278/2553] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:32.405995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.415225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb68539-8369-4be7-b99f-ab0fb124ff1a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.419971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.420640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.420914 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.421180 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.421892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:32.422189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.423756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.434939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1276/2554] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:51:32.439297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.448313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9c0ea-0d27-49b1-80bf-6d4dbafbcf2e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.453760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.454411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.454696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.454973 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.455659 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.457568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.482832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1279/2555] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:32.486984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.497740 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa77698-84c1-48c1-b84f-057134e43223 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.560427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.561104 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.561387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.561662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.562370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.564289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.584730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1277/2556] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:32.588584 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.599154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc1c15-588c-43d9-ab31-0dd0714f5f3e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.603764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.604442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.604727 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.605034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.605760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.607333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.617832 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1280/2557] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:32.621931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.631949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936020d-7362-4797-8eda-f5f445a53cbe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.637336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4004ae-1c55-4811-ad96-6f53230082d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.637985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4004ae-1c55-4811-ad96-6f53230082d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.638265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4004ae-1c55-4811-ad96-6f53230082d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.638538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4004ae-1c55-4811-ad96-6f53230082d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.639258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4004ae-1c55-4811-ad96-6f53230082d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.640875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4004ae-1c55-4811-ad96-6f53230082d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.650596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1278/2558] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:32.654647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.664443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b14397-2101-42fe-b888-3db73f68004f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.669249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.670108 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.670404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.670655 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] PATH_INFO: `/v3/projects/413d11bcd4fb4bbea66fcdd11f5bd3ca/users/9dc2bb8dffc6428fae2c308b8a99cd4f/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.685050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=413d11bcd4fb4bbea66fcdd11f5bd3ca, user_id=9dc2bb8dffc6428fae2c308b8a99cd4f, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.687675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.702289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1281/2559] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:32 2022] PUT /identity/v3/projects/413d11bcd4fb4bbea66fcdd11f5bd3ca/users/9dc2bb8dffc6428fae2c308b8a99cd4f/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:32.712490 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.712952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.713070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.738705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] MFA Rules not processed for user `9dc2bb8dffc6428fae2c308b8a99cd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:32.739885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:32.801638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1279/2560] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:32.806029 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.864662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e88b5-5e38-4119-a18b-65237fc858a4 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.869258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.869945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.870224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.870568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.871866 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.873764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.898030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1282/2561] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:32.902457 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:32.912924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f37f9-6a28-43d6-8afe-a731cd318b8f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:32.970204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:32.970854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:32.971129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:32.971398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:32.972074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:32.973984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:32.993058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1280/2562] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:32 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:32.997778 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:33.008035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811515a-e8c6-431e-9a6b-85d9bee1f4fb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:33.012865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:33.013512 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.013804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.014082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.014802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:33.016466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:33.026837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1283/2563] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:33.030932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:33.040386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944847-8297-4e20-9bc0-edf3b648617d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:33.045708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:33.046562 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.046839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.047112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] PATH_INFO: `/v3/projects/dbd0c7b76b544eb0a348fab0f1edc0de/users/2c8a22552ef84515b8641341f99e9035/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.062095 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd0c7b76b544eb0a348fab0f1edc0de, user_id=2c8a22552ef84515b8641341f99e9035, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:33.064682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:33.077582 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1281/2564] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:33 2022] PUT /identity/v3/projects/dbd0c7b76b544eb0a348fab0f1edc0de/users/2c8a22552ef84515b8641341f99e9035/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:33.081451 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:33.091228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc92a7-6ed4-46fa-af00-82e3277c7e6f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:33.144995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f189afb-00ed-42eb-a074-554d3c631748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:33.145651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f189afb-00ed-42eb-a074-554d3c631748 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.145935 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f189afb-00ed-42eb-a074-554d3c631748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.146212 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f189afb-00ed-42eb-a074-554d3c631748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.147050 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f189afb-00ed-42eb-a074-554d3c631748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:33.148945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f189afb-00ed-42eb-a074-554d3c631748 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:33.159628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1284/2565] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:33.163664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:33.174514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ac78e-5f3a-4983-98e7-2e8cb0f6b1f1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:33.179731 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:33.180651 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.180972 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.181244 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] PATH_INFO: `/v3/projects/dbd0c7b76b544eb0a348fab0f1edc0de/users/2c8a22552ef84515b8641341f99e9035/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.184036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd0c7b76b544eb0a348fab0f1edc0de, user_id=2c8a22552ef84515b8641341f99e9035, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:33.186695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:33.196825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1282/2566] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:33 2022] PUT /identity/v3/projects/dbd0c7b76b544eb0a348fab0f1edc0de/users/2c8a22552ef84515b8641341f99e9035/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:33.208382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.208886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.209000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.235355 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] MFA Rules not processed for user `2c8a22552ef84515b8641341f99e9035`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:33.236575 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:33.301190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1285/2567] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:33.311590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:33.313415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:33.368561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce49ba0-d79e-4d63-8a00-2ed7a9e92518 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:33.372475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:33.373105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.373381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.373656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.382501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:33.383910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:33.393196 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1283/2568] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:33.488024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:33.498016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a747fe-f550-424e-afa4-1e3045c86a2e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:33.501648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82469d9-8985-448f-a171-1e8ac770aa02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:33.502283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82469d9-8985-448f-a171-1e8ac770aa02 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:33.502568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82469d9-8985-448f-a171-1e8ac770aa02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:33.502847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82469d9-8985-448f-a171-1e8ac770aa02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:33.558145 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82469d9-8985-448f-a171-1e8ac770aa02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:33.559684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82469d9-8985-448f-a171-1e8ac770aa02 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:33.568409 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1286/2569] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:36.114571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:36.174674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47cdf6-fe1b-4858-95d5-94b80e234895 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:36.179187 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:36.180044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:36.180354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:36.180631 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] PATH_INFO: `/v3/users/9dc2bb8dffc6428fae2c308b8a99cd4f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:36.181806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc2bb8dffc6428fae2c308b8a99cd4f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:36.183712 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:36.229535 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc2bb8dffc6428fae2c308b8a99cd4f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.236666 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc2bb8dffc6428fae2c308b8a99cd4f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.244093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc2bb8dffc6428fae2c308b8a99cd4f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.244730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc2bb8dffc6428fae2c308b8a99cd4f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.256453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc2bb8dffc6428fae2c308b8a99cd4f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.262072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1284/2570] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:36 2022] DELETE /identity/v3/users/9dc2bb8dffc6428fae2c308b8a99cd4f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:36.266577 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:36.267085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:36.267188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:36.284739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:36.285927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:36.341661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1287/2571] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:36.351156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:36.352569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:36.409706 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a302-75e0-45ec-83ed-1f186963a860 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:36.413296 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:36.413926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:36.414204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:36.414473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:36.423796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:36.425239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:36.434320 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1285/2572] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:36.695538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:36.697003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:36.710005 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3d9a4-d4e0-4b0b-b51a-76c4bee9b3e6 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:36.715182 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:36.717206 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:36.717491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:36.717767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] PATH_INFO: `/v3/projects/413d11bcd4fb4bbea66fcdd11f5bd3ca` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:36.719054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=413d11bcd4fb4bbea66fcdd11f5bd3ca)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:36.720913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:36.768649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413d11bcd4fb4bbea66fcdd11f5bd3ca', 'request_id': 'req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.775552 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413d11bcd4fb4bbea66fcdd11f5bd3ca', 'request_id': 'req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.782003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413d11bcd4fb4bbea66fcdd11f5bd3ca', 'request_id': 'req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.793303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] The token cache is being invalidate because project 413d11bcd4fb4bbea66fcdd11f5bd3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:36.793629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.794279 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.795808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1288/2573] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:36 2022] DELETE /identity/v3/projects/413d11bcd4fb4bbea66fcdd11f5bd3ca => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:36.799961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:36.809308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91716c43-90e2-412f-bd05-f810287cd38c service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:36.864871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:36.865721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:36.865994 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:36.866259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] PATH_INFO: `/v3/users/2c8a22552ef84515b8641341f99e9035` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:36.867555 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8a22552ef84515b8641341f99e9035)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:36.869422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:36.915835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8a22552ef84515b8641341f99e9035'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.922819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8a22552ef84515b8641341f99e9035'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.931139 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8a22552ef84515b8641341f99e9035'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.931799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8a22552ef84515b8641341f99e9035'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.944902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82e91c15-7f9e-4acc-9021-c2fb900a5dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8a22552ef84515b8641341f99e9035'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:36.952410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1286/2574] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:36 2022] DELETE /identity/v3/users/2c8a22552ef84515b8641341f99e9035 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:37.214048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.215535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:37.228489 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361dffd7-3fd7-41ea-b83f-ff00df3e5d9f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.285667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.286783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.286783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.287124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] PATH_INFO: `/v3/projects/dbd0c7b76b544eb0a348fab0f1edc0de` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.288286 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd0c7b76b544eb0a348fab0f1edc0de)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.290203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.337951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd0c7b76b544eb0a348fab0f1edc0de', 'request_id': 'req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:37.345069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd0c7b76b544eb0a348fab0f1edc0de', 'request_id': 'req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:37.350219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd0c7b76b544eb0a348fab0f1edc0de', 'request_id': 'req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:37.361446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] The token cache is being invalidate because project dbd0c7b76b544eb0a348fab0f1edc0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:37.361729 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:37.362969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df6eb1f3-7c0e-46d1-8022-5f114a1ce346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:37.364522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1289/2575] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:37 2022] DELETE /identity/v3/projects/dbd0c7b76b544eb0a348fab0f1edc0de => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:37.369554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.370045 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.370147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.388625 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:37.389800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:37.446370 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1287/2576] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:37.454792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.455089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.455450 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.473556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:37.474752 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:37.483237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1290/2577] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:37.487444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.497587 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d99f5e-a68f-498d-98f3-86c0e1283c86 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.502018 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.502653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.502932 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.503493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.504249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:37.504549 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.506127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.517443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1288/2578] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:51:37.521732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.530960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324b0b2-b4e6-4a0f-9b4d-b84a15db368a None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.535581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.536280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.536567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.536846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.537518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.539413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.564891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1291/2579] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:37 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:37.568926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.578416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e14a9-cd1a-40e2-b7bd-ac6b6ea6ecaa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.635010 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.635680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.635962 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.636269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.636982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.639383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.659873 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1289/2580] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:37 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:37.663981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.674664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35dbc4-4df3-4fb2-9184-7886520e3fd9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.682202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.683202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.683712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.684768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.686062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.688506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.703581 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1292/2581] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:37.708142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.718385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274895f-994c-4eca-b193-dc96ee0509af admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.722967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8701d4f-39b4-441e-820d-85286d30dd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.723607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701d4f-39b4-441e-820d-85286d30dd63 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.723883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701d4f-39b4-441e-820d-85286d30dd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.724152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701d4f-39b4-441e-820d-85286d30dd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.724904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8701d4f-39b4-441e-820d-85286d30dd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.726680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8701d4f-39b4-441e-820d-85286d30dd63 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.737648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1290/2582] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:37.741769 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.752761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4800e-acae-4cd1-b80a-715f830a28ba admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:37.757690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:37.758546 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.758826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.759097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] PATH_INFO: `/v3/projects/4e24393b2f0942b4829bfb4bf7b9318c/users/2fc5d9137b624b8992c69959da51868f/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.773643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e24393b2f0942b4829bfb4bf7b9318c, user_id=2fc5d9137b624b8992c69959da51868f, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:37.776294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:37.788373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1293/2583] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:37 2022] PUT /identity/v3/projects/4e24393b2f0942b4829bfb4bf7b9318c/users/2fc5d9137b624b8992c69959da51868f/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:37.800728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:37.801014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:37.801289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:37.832645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] MFA Rules not processed for user `2fc5d9137b624b8992c69959da51868f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:37.834497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:37.908657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1291/2584] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:37.913154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:37.995241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4bb6b-c484-4b71-a83f-f5ca16d8c5d3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.002671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.003968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.004486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.004964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.006179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.010141 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.042799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1294/2585] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:37 2022] POST /identity/v3/projects => generated 409 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:38.048212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.060642 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d35882-4b42-4b54-a7f3-7e50cb298af6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.123340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.124016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.124338 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.124616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.125480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.127214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.152583 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1292/2586] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:38 2022] POST /identity/v3/users => generated 311 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:38.154052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.165868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d748c77-bbdf-4319-ab0e-4a7e617d1002 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.170845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.171513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.171805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.172123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.172974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.175118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.187631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1295/2587] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:38.192785 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.204040 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1b9a-2949-4a18-8aa1-ad6ff393ed66 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.209023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.209931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.210203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.210500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] PATH_INFO: `/v3/projects/578eaede9103464aa39d6a68f60a97b0/users/cbac865bab1241b8ac7f9b5231b0f59c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.226383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578eaede9103464aa39d6a68f60a97b0, user_id=cbac865bab1241b8ac7f9b5231b0f59c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.232333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.250826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1293/2588] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:38 2022] PUT /identity/v3/projects/578eaede9103464aa39d6a68f60a97b0/users/cbac865bab1241b8ac7f9b5231b0f59c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:38.255373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.268863 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dc8d0-b12d-443c-bd39-f1049c6ac628 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.333679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf90b9a-67be-4216-a9e6-fc80ebe1e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.334733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf90b9a-67be-4216-a9e6-fc80ebe1e502 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.334733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf90b9a-67be-4216-a9e6-fc80ebe1e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.335003 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf90b9a-67be-4216-a9e6-fc80ebe1e502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.335798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf90b9a-67be-4216-a9e6-fc80ebe1e502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.337544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf90b9a-67be-4216-a9e6-fc80ebe1e502 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.348346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1296/2589] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:38.352787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.364933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c414152-af9b-4f40-945c-0172ee405b71 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.369872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.370826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.371168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.371507 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] PATH_INFO: `/v3/projects/578eaede9103464aa39d6a68f60a97b0/users/cbac865bab1241b8ac7f9b5231b0f59c/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.374600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578eaede9103464aa39d6a68f60a97b0, user_id=cbac865bab1241b8ac7f9b5231b0f59c, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.377398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.389381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1294/2590] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:38 2022] PUT /identity/v3/projects/578eaede9103464aa39d6a68f60a97b0/users/cbac865bab1241b8ac7f9b5231b0f59c/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:38.401732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.402111 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.402476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.432390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] MFA Rules not processed for user `cbac865bab1241b8ac7f9b5231b0f59c`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:38.433847 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:38.529251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1297/2591] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:38.541825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.544353 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:38.603475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a26ba-0b8c-4763-bd2f-d7aa2193bde3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.607094 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.607722 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.607998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.608305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.617167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.618590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.629219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1295/2592] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:51:38.771861 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:38.782301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7be66e-9bd3-4bc6-ad61-a9021f11cb8c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:38.785932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb798d-2fd0-4468-88c1-37e7527f5afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:38.786567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb798d-2fd0-4468-88c1-37e7527f5afd service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:38.786854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb798d-2fd0-4468-88c1-37e7527f5afd service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:38.787133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb798d-2fd0-4468-88c1-37e7527f5afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:38.843322 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb798d-2fd0-4468-88c1-37e7527f5afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:38.844774 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb798d-2fd0-4468-88c1-37e7527f5afd service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:38.853619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1298/2593] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:49.185780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:49.244835 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380f143-2ca4-4a3f-b527-1eb6bb811007 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:49.249406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:49.250262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:49.250545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:49.250820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] PATH_INFO: `/v3/users/2fc5d9137b624b8992c69959da51868f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:49.251982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc5d9137b624b8992c69959da51868f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:49.254194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:49.299885 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc5d9137b624b8992c69959da51868f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:49.306220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc5d9137b624b8992c69959da51868f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:49.315250 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc5d9137b624b8992c69959da51868f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:49.315969 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc5d9137b624b8992c69959da51868f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:49.328152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc5d9137b624b8992c69959da51868f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:49.334210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1296/2594] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:49 2022] DELETE /identity/v3/users/2fc5d9137b624b8992c69959da51868f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:49.338924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:49.339428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:49.339542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:49.358122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:49.359365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:49.417186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1299/2595] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:49.427686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:49.429206 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:49.490066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82932b58-9a4c-400a-9168-73f9135956be admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:49.494818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:49.495797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:49.496198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:49.496543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:49.507272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:49.508790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:49.519090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1297/2596] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:50.012405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.014677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.026521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0b3a5-80a8-4e39-9ebe-dc3ee457c2bd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.031736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.032634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.032900 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.033173 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] PATH_INFO: `/v3/projects/4e24393b2f0942b4829bfb4bf7b9318c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.034353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e24393b2f0942b4829bfb4bf7b9318c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.036287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.081198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e24393b2f0942b4829bfb4bf7b9318c', 'request_id': 'req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.088015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e24393b2f0942b4829bfb4bf7b9318c', 'request_id': 'req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.093843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e24393b2f0942b4829bfb4bf7b9318c', 'request_id': 'req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.104046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] The token cache is being invalidate because project 4e24393b2f0942b4829bfb4bf7b9318c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:50.104353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.104968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.106472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1300/2597] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:50 2022] DELETE /identity/v3/projects/4e24393b2f0942b4829bfb4bf7b9318c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:50.110500 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.119760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83529c0-b7d4-4800-bae6-6a8b6252e0dd service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.173442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.174299 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.174579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.174854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] PATH_INFO: `/v3/users/cbac865bab1241b8ac7f9b5231b0f59c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.176023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbac865bab1241b8ac7f9b5231b0f59c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.177955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.221330 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbac865bab1241b8ac7f9b5231b0f59c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.227630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbac865bab1241b8ac7f9b5231b0f59c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.234760 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbac865bab1241b8ac7f9b5231b0f59c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.235371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbac865bab1241b8ac7f9b5231b0f59c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.246767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbac865bab1241b8ac7f9b5231b0f59c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.252803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1298/2598] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:50 2022] DELETE /identity/v3/users/cbac865bab1241b8ac7f9b5231b0f59c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:50.514700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.516767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.534223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4642c2a-0d8e-4056-b06b-7cf15ec4a73c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.534481 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.536119 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.606919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.608275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.608616 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.608824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] PATH_INFO: `/v3/projects/578eaede9103464aa39d6a68f60a97b0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.610037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578eaede9103464aa39d6a68f60a97b0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.611958 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.612525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc8145-fa79-42de-b725-c934fbf707de admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.622324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.624134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.624611 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.625037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] PATH_INFO: `/v3/users/bf1b31c3bf144ef0bb3a932b14eeab17` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.626509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1b31c3bf144ef0bb3a932b14eeab17)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.628942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.658920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578eaede9103464aa39d6a68f60a97b0', 'request_id': 'req-790f8797-d03f-4ba3-b968-99a639010b14'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.666196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578eaede9103464aa39d6a68f60a97b0', 'request_id': 'req-790f8797-d03f-4ba3-b968-99a639010b14'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.672124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578eaede9103464aa39d6a68f60a97b0', 'request_id': 'req-790f8797-d03f-4ba3-b968-99a639010b14'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.674889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1b31c3bf144ef0bb3a932b14eeab17'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.681270 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1b31c3bf144ef0bb3a932b14eeab17'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.682682 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] The token cache is being invalidate because project 578eaede9103464aa39d6a68f60a97b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:50.682968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.683618 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-790f8797-d03f-4ba3-b968-99a639010b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.685735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1301/2599] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:50 2022] DELETE /identity/v3/projects/578eaede9103464aa39d6a68f60a97b0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:50.689965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1b31c3bf144ef0bb3a932b14eeab17'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.690644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1b31c3bf144ef0bb3a932b14eeab17'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.691630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83082337-ff23-4949-b0ea-8a5687093afe None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.691919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83082337-ff23-4949-b0ea-8a5687093afe None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.692251 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83082337-ff23-4949-b0ea-8a5687093afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.702839 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-356a78c1-f882-4b67-ae9f-5dfdafa13d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1b31c3bf144ef0bb3a932b14eeab17'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:50.708858 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1299/2600] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:51:50 2022] DELETE /identity/v3/users/bf1b31c3bf144ef0bb3a932b14eeab17 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:50.710256 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-83082337-ff23-4949-b0ea-8a5687093afe None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:50.711536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-83082337-ff23-4949-b0ea-8a5687093afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.713054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.713346 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.713623 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.730239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:50.731424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.769240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1302/2601] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:50.776985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.777274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.777548 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.790478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1300/2602] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:50.796234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:50.797445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.800413 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.801703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:50.805831 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1303/2603] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:50.810068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.819313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948ce1a-1134-4b1f-b694-5cf7e47fb3a8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.824039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.824787 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.825071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.825348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.827076 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:50.827393 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.829021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.844208 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1304/2604] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:51:50.845435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.854687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbabdd-e3bf-4c5f-9065-e4f6a6ebba1b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.856273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4d005-f6c7-43f7-a56b-7344cc6a151e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.859706 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.859880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.860373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.860509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.860689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.860807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.860968 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.861083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.861646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.863508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.869781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.871170 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.881068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1301/2605] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:51:50.892601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1305/2606] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:50 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:51:50.893779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:50.903247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87986e5b-3c89-42da-8d49-a0452931f835 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:50.963526 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:50.964227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:50.964515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:50.964791 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:50.965475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:50.968308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:50.991600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1302/2607] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:50 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:50.993038 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.003643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b5ed-2619-4548-ae97-09b3b9e91eeb admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.008287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.008947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.009231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.009507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.010239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.011802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.026075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1306/2608] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:51.029450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.039551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bff164-ada1-444d-b4e1-01ac1fb4cf38 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.044397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcd4bc-6046-4abf-9187-4509be288ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.045134 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcd4bc-6046-4abf-9187-4509be288ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.045421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcd4bc-6046-4abf-9187-4509be288ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.045802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcd4bc-6046-4abf-9187-4509be288ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.046590 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcd4bc-6046-4abf-9187-4509be288ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.048312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcd4bc-6046-4abf-9187-4509be288ccb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.060218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1303/2609] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:51.062679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.073048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45b17b-9d95-4c2a-a4ae-c7631f7aa318 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.078974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.079860 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.080142 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.080449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] PATH_INFO: `/v3/projects/a951ccd6d9a142c0b187028edc55c3bc/users/ee130a8479a64c589648ef35fd89dc50/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.095716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a951ccd6d9a142c0b187028edc55c3bc, user_id=ee130a8479a64c589648ef35fd89dc50, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.098459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.111845 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1307/2610] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:51 2022] PUT /identity/v3/projects/a951ccd6d9a142c0b187028edc55c3bc/users/ee130a8479a64c589648ef35fd89dc50/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:51.116992 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.117283 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.117609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.135468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:51.136703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.155327 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.157127 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.205057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1304/2611] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:51.214498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.215862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.229216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1034fe-6ba6-475f-a6df-7f4e3cbde789 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.234798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.235649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.235931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.236303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] PATH_INFO: `/v3/projects/53e97d53681a48b8a7a3de276d87f85d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.237486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e97d53681a48b8a7a3de276d87f85d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.239388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.273769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dac602-8a48-48e4-89f5-dede40be9bb3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.286782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e97d53681a48b8a7a3de276d87f85d', 'request_id': 'req-558fbb58-9149-4baa-bffa-c19ffc2156ef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:51.293901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e97d53681a48b8a7a3de276d87f85d', 'request_id': 'req-558fbb58-9149-4baa-bffa-c19ffc2156ef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:51.299085 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e97d53681a48b8a7a3de276d87f85d', 'request_id': 'req-558fbb58-9149-4baa-bffa-c19ffc2156ef'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:51.310646 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] The token cache is being invalidate because project 53e97d53681a48b8a7a3de276d87f85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:51:51.311345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:51.311704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-558fbb58-9149-4baa-bffa-c19ffc2156ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:51:51.313295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1308/2612] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:51:51 2022] DELETE /identity/v3/projects/53e97d53681a48b8a7a3de276d87f85d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:51.318082 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9dba2-c3a6-4338-ae14-65c303598e50 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.318371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9dba2-c3a6-4338-ae14-65c303598e50 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.318667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9dba2-c3a6-4338-ae14-65c303598e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.324042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.324716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.324986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.325256 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.327663 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.337406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4ff9dba2-c3a6-4338-ae14-65c303598e50 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:51.338722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4ff9dba2-c3a6-4338-ae14-65c303598e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.390545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.391996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.396570 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1309/2613] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:51.401775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1305/2614] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:51:51.404755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.405048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.405329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.424077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:51.425904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.434198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1310/2615] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:51.438680 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.447928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed04701-97be-4be7-8de0-7f34ea248bf1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.452626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.453276 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.453559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.453834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.454553 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:51:51.454860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.456470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.472475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1306/2616] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:51:51.474093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.484298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5902b-0f0e-4f3e-b404-62df58b67eba None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.489130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.489775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.490055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.490325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.491002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.493146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.520679 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1311/2617] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:51 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:51.525093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.536198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c2fab-6709-410d-a1da-9d95443ee658 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.595482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.596151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.596455 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.596728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.597422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.599324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.620422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1307/2618] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:51 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:51.624634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.636786 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102c7e-9c2f-4f72-b4d5-df995d831037 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.642002 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.642673 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.642961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.643241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.643967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.645601 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.656495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1312/2619] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:51.660957 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.671897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177eb8e-0271-4b28-9dec-a342cd8c99b3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.680265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b49fa4-cf84-4710-8bc2-8d450b6f39c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.680999 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b49fa4-cf84-4710-8bc2-8d450b6f39c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.681259 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b49fa4-cf84-4710-8bc2-8d450b6f39c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.681539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b49fa4-cf84-4710-8bc2-8d450b6f39c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.682275 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b49fa4-cf84-4710-8bc2-8d450b6f39c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.683874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b49fa4-cf84-4710-8bc2-8d450b6f39c2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.699030 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1308/2620] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:51:51.700281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.711325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0c432-9aa1-4e31-bf51-4f1ab62a99a6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.716969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.718495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.718782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.719065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] PATH_INFO: `/v3/projects/583159e196b2438698885193ca03ef4d/users/7cee74fc982d4aa39f56c6b40de259cc/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.735300 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] RBAC: Authorizing `identity:create_grant(project_id=583159e196b2438698885193ca03ef4d, user_id=7cee74fc982d4aa39f56c6b40de259cc, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.738016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.754476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1313/2621] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:51 2022] PUT /identity/v3/projects/583159e196b2438698885193ca03ef4d/users/7cee74fc982d4aa39f56c6b40de259cc/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:51:51.759461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357f80a-948b-4bed-8b78-a961721956d3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.760097 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357f80a-948b-4bed-8b78-a961721956d3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.760629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357f80a-948b-4bed-8b78-a961721956d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.788695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7357f80a-948b-4bed-8b78-a961721956d3 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:51.790009 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7357f80a-948b-4bed-8b78-a961721956d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.855820 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1309/2622] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:51.865452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:51.867498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:51.931259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b8664-9d42-41e2-8957-16a33dcd259e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:51.935670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:51.936476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:51.936628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:51.936904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:51.946061 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:51.947482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:51.957726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1314/2623] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:52.051483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.051788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.052814 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.085141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] MFA Rules not processed for user `ee130a8479a64c589648ef35fd89dc50`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:52.087029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:52.167293 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1310/2624] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:52.172032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:52.173535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:52.189276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffb43-5118-4442-bf72-36a80a222fdf service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:52.195511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:52.196928 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.197398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.197830 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.198722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:52.201636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:52.234663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1315/2625] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:52 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:52.239029 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:52.252646 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f4305-6f2c-4770-9472-a665b174fd0c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:52.318478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:52.319171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.319387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.319597 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.320313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:52.322477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:52.346073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1311/2626] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:52 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:52.349621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:52.360567 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10442b-a869-4b1c-acc8-ba30f77f5e36 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:52.365874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:52.366612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.366976 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.367316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.368104 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:52.369814 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:52.381943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1316/2627] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:52.385777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:52.396301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a450c8-9663-426d-8350-129c34df0a50 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:52.401285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:52.402715 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.403236 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.403607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] PATH_INFO: `/v3/projects/e61255d9a2f04207b2e34b68efa6c904/users/6d8e8fced33b497cb23b2f1c3ec38f96/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.421106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61255d9a2f04207b2e34b68efa6c904, user_id=6d8e8fced33b497cb23b2f1c3ec38f96, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:52.423771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:52.437042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1312/2628] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:52 2022] PUT /identity/v3/projects/e61255d9a2f04207b2e34b68efa6c904/users/6d8e8fced33b497cb23b2f1c3ec38f96/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:52.441411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:52.452058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edee09-31b4-4ee5-a6de-2d1af7f4103a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:52.507804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43835a15-b5a9-41c6-b27b-860f5f25484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:52.508542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43835a15-b5a9-41c6-b27b-860f5f25484c admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.508849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43835a15-b5a9-41c6-b27b-860f5f25484c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.509043 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43835a15-b5a9-41c6-b27b-860f5f25484c admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.509779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43835a15-b5a9-41c6-b27b-860f5f25484c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:52.511311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43835a15-b5a9-41c6-b27b-860f5f25484c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:52.521991 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1317/2629] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:52.526654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:52.537737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7719d-0599-421e-9373-87fc7c4ef1af admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:52.542434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:52.543382 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:52.543579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:52.543849 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] PATH_INFO: `/v3/projects/e61255d9a2f04207b2e34b68efa6c904/users/6d8e8fced33b497cb23b2f1c3ec38f96/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:52.546471 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61255d9a2f04207b2e34b68efa6c904, user_id=6d8e8fced33b497cb23b2f1c3ec38f96, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:52.549150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:52.559804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1313/2630] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:52 2022] PUT /identity/v3/projects/e61255d9a2f04207b2e34b68efa6c904/users/6d8e8fced33b497cb23b2f1c3ec38f96/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:53.686750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:53.687213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:53.687295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:53.727759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] MFA Rules not processed for user `6d8e8fced33b497cb23b2f1c3ec38f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:53.729297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:53.806313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1318/2631] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:51:53.818865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:53.820656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:53.884278 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e7858-6211-4f20-922d-abd09a994056 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:53.889703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f4b07-baf0-419b-98fe-46e561664de1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:53.890667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f4b07-baf0-419b-98fe-46e561664de1 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:53.891156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f4b07-baf0-419b-98fe-46e561664de1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:53.891606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f4b07-baf0-419b-98fe-46e561664de1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:53.963386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f4b07-baf0-419b-98fe-46e561664de1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:53.965028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f4b07-baf0-419b-98fe-46e561664de1 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:53.976032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1314/2632] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:51:55.455721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:55.457805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:55.473035 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd8b48-2dd4-4d32-b5df-f814fbdbea84 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:55.479042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:55.479916 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:55.480437 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:55.480808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:55.491388 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:55.493314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:55.504222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1319/2633] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:51:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:51:56.561165 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:56.562047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:56.562331 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:56.594737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] MFA Rules not processed for user `7cee74fc982d4aa39f56c6b40de259cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:51:56.595883 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:51:56.665698 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1315/2634] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:51:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:51:56.670979 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:56.736760 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572fe45-39e5-4e23-9f85-269f7207071e service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:56.741276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:56.741965 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:56.742377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:56.742651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:56.743344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:56.745289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:56.776397 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1320/2635] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:51:56 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:56.777695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:56.789380 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e657a0-545e-4cae-bde6-f5361497cb8b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:56.852789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:56.854237 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:56.854714 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:56.855450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:56.856399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:56.858738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:56.882020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1316/2636] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:51:56 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:51:56.886057 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:56.910194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da640053-2713-4352-a8ee-7543e65852d5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:56.915901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:56.916857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:56.917333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:56.917822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:56.918746 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:56.920897 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:56.935605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1321/2637] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:56.940541 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:56.954310 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b333fb-0ec3-4353-a65d-eb5eeb976a43 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:56.961127 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:56.962013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:56.962350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:56.962621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] PATH_INFO: `/v3/projects/72c4c25ece984355b0f8eb63e27c99b8/users/8ccbd90986a34e6ba1e74df6c56aa386/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:56.985101 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c4c25ece984355b0f8eb63e27c99b8, user_id=8ccbd90986a34e6ba1e74df6c56aa386, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:56.988696 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:57.021478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1317/2638] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:56 2022] PUT /identity/v3/projects/72c4c25ece984355b0f8eb63e27c99b8/users/8ccbd90986a34e6ba1e74df6c56aa386/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:51:57.025233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:57.038122 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1bfe8-a17b-4042-ae76-df1a67ccc300 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:57.106672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e182e64-8252-477c-8c78-889d4d93e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:57.108012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e182e64-8252-477c-8c78-889d4d93e83e admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:57.108336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e182e64-8252-477c-8c78-889d4d93e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:57.108637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e182e64-8252-477c-8c78-889d4d93e83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:57.109497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e182e64-8252-477c-8c78-889d4d93e83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:57.118149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e182e64-8252-477c-8c78-889d4d93e83e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:57.135428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1322/2639] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:51:57.136475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:51:57.162015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782436a-99e3-41c4-b11d-7cac8fdc03a4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:51:57.170729 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:51:57.171617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:51:57.171892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:51:57.172190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] PATH_INFO: `/v3/projects/72c4c25ece984355b0f8eb63e27c99b8/users/8ccbd90986a34e6ba1e74df6c56aa386/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:51:57.174264 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c4c25ece984355b0f8eb63e27c99b8, user_id=8ccbd90986a34e6ba1e74df6c56aa386, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:51:57.179077 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:51:57.200843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1318/2640] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:51:57 2022] PUT /identity/v3/projects/72c4c25ece984355b0f8eb63e27c99b8/users/8ccbd90986a34e6ba1e74df6c56aa386/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:52:04.167248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce90399-70af-463d-9f86-f3eecc022c47 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:04.167671 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce90399-70af-463d-9f86-f3eecc022c47 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:04.167799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce90399-70af-463d-9f86-f3eecc022c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:04.199000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-bce90399-70af-463d-9f86-f3eecc022c47 None None] MFA Rules not processed for user `8ccbd90986a34e6ba1e74df6c56aa386`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:04.200643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-bce90399-70af-463d-9f86-f3eecc022c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:04.282960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1323/2641] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:04.297394 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:04.299319 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:04.373981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed9bd-582d-47c2-a598-afa4209d3706 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:04.379542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:04.380203 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:04.380477 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:04.380746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:04.439368 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:04.441710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:04.450937 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1319/2642] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:05.491302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:05.491884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:05.491884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:05.513588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] MFA Rules not processed for user `7cee74fc982d4aa39f56c6b40de259cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:05.514820 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:05.531978 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1324/2643] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:05.854978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:05.933893 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263f59-dc9b-4430-8baa-2c6a5e2a8f64 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:05.940334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:05.941054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:05.941844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:05.941844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:05.943630 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:06.048355 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:06.050515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:06.069732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1320/2644] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:09.899069 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:09.900808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:09.924930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720b4f-1377-46f3-aaed-e032254c5b5e None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:09.933942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:09.935641 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:09.935931 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:09.936269 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] PATH_INFO: `/v3/users/7a2b51125e9645dc9454042f49ff3659` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:09.937911 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2b51125e9645dc9454042f49ff3659)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:09.940201 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:09.999764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2b51125e9645dc9454042f49ff3659'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.015342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2b51125e9645dc9454042f49ff3659'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.016136 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2b51125e9645dc9454042f49ff3659'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.016817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2b51125e9645dc9454042f49ff3659'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.025971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2b51125e9645dc9454042f49ff3659'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.031850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.043943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cb8e4-6d19-49e6-bf28-f6db73b979aa service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.047782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1325/2645] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:52:09 2022] DELETE /identity/v3/users/7a2b51125e9645dc9454042f49ff3659 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:10.051546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.052456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.052734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.053200 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] PATH_INFO: `/v3/users/ee130a8479a64c589648ef35fd89dc50` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.054723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee130a8479a64c589648ef35fd89dc50)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.056665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.109794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee130a8479a64c589648ef35fd89dc50'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.118650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee130a8479a64c589648ef35fd89dc50'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.127080 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee130a8479a64c589648ef35fd89dc50'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.127721 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee130a8479a64c589648ef35fd89dc50'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.144294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee130a8479a64c589648ef35fd89dc50'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.151168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1321/2646] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:52:10 2022] DELETE /identity/v3/users/ee130a8479a64c589648ef35fd89dc50 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:10.266531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.267942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.384021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0485f-25e4-41e7-a88c-4b9fbc5fe674 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.388376 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.395348 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.455192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.456058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.456390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.456670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] PATH_INFO: `/v3/projects/bdb0d817bd8e4eeca5ed6229a4b5c53a` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.460213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb0d817bd8e4eeca5ed6229a4b5c53a)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.462513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.473333 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f7f21-e8ad-4875-a080-d2989be78316 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.481167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.482734 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.483235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.483797 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] PATH_INFO: `/v3/projects/a951ccd6d9a142c0b187028edc55c3bc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.485852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a951ccd6d9a142c0b187028edc55c3bc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.488728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.513886 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb0d817bd8e4eeca5ed6229a4b5c53a', 'request_id': 'req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.523734 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb0d817bd8e4eeca5ed6229a4b5c53a', 'request_id': 'req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.531137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb0d817bd8e4eeca5ed6229a4b5c53a', 'request_id': 'req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.540871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a951ccd6d9a142c0b187028edc55c3bc', 'request_id': 'req-4509f09c-2552-42f8-801f-8fc957d55484'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.542295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] The token cache is being invalidate because project bdb0d817bd8e4eeca5ed6229a4b5c53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:52:10.542587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.543223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d4d8a21f-1c1f-4f6c-886b-5aeb729a54a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.545880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1326/2647] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:52:10 2022] DELETE /identity/v3/projects/bdb0d817bd8e4eeca5ed6229a4b5c53a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:10.547512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a951ccd6d9a142c0b187028edc55c3bc', 'request_id': 'req-4509f09c-2552-42f8-801f-8fc957d55484'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.551015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc33af-244f-4105-b8dc-c3c6be1b080f None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.551376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc33af-244f-4105-b8dc-c3c6be1b080f None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.551722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc33af-244f-4105-b8dc-c3c6be1b080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.553087 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a951ccd6d9a142c0b187028edc55c3bc', 'request_id': 'req-4509f09c-2552-42f8-801f-8fc957d55484'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.566882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] The token cache is being invalidate because project a951ccd6d9a142c0b187028edc55c3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:52:10.567172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.567926 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.569533 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1322/2648] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:52:10 2022] DELETE /identity/v3/projects/a951ccd6d9a142c0b187028edc55c3bc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:10.573398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.573545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-eddc33af-244f-4105-b8dc-c3c6be1b080f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:10.574686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-eddc33af-244f-4105-b8dc-c3c6be1b080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.574882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.639406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1327/2649] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:10.642298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509f09c-2552-42f8-801f-8fc957d55484 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.647913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.648495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.649046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.650821 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.651915 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.652294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.652635 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] PATH_INFO: `/v3/users/6d8e8fced33b497cb23b2f1c3ec38f96` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.654899 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8e8fced33b497cb23b2f1c3ec38f96)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.656950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.670672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:10.671938 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.685924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1328/2650] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:10.690371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.700696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411ac9a-2d91-472d-9cf1-907cbe486c2d None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.706063 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.706766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.707064 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.707329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.708066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:52:10.708701 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.712282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.715357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8e8fced33b497cb23b2f1c3ec38f96'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.723765 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8e8fced33b497cb23b2f1c3ec38f96'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.727040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1329/2651] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:52:10.734432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8e8fced33b497cb23b2f1c3ec38f96'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.736597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.738851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8e8fced33b497cb23b2f1c3ec38f96'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.748318 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5dae6-fece-47e4-9fc0-941e8b1c4afd admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.752185 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8e8fced33b497cb23b2f1c3ec38f96'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:10.753143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.754219 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.754500 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.754776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.755463 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.757547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.760513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1323/2652] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:52:10 2022] DELETE /identity/v3/users/6d8e8fced33b497cb23b2f1c3ec38f96 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:10.787331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1330/2653] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:52:10 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:52:10.791111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.792589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.859640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83f3ee-ac61-4b4d-a7eb-e796d6352262 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.915545 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.916285 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.916594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.916871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.917572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.920295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.941827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1324/2654] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:52:10 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:52:10.946864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.951109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.952917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:10.958837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59226ebc-c3e2-4c86-8f91-1c222dd95639 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.963914 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.964638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.965096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.965373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.965888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8789f-f5d1-4339-be0e-cf8c20fcf1af admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:10.966712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.968374 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.975235 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:10.976233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:10.976589 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:10.976941 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] PATH_INFO: `/v3/projects/e61255d9a2f04207b2e34b68efa6c904` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:10.978207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e61255d9a2f04207b2e34b68efa6c904)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:10.978996 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1331/2655] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:52:10.980882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:10.984068 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:10.994574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc6106-c8b5-4f41-b353-f7f697356b98 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.003225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.004193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.004477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.004748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.005476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.007054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.018626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1332/2656] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:52:11.022839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.030424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e61255d9a2f04207b2e34b68efa6c904', 'request_id': 'req-08e3d405-d1d4-4d98-93b2-447515624808'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:11.033656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30010d4-2e17-48b5-b012-d824901ffaca admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.038550 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e61255d9a2f04207b2e34b68efa6c904', 'request_id': 'req-08e3d405-d1d4-4d98-93b2-447515624808'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:11.043861 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e61255d9a2f04207b2e34b68efa6c904', 'request_id': 'req-08e3d405-d1d4-4d98-93b2-447515624808'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:11.055900 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] The token cache is being invalidate because project e61255d9a2f04207b2e34b68efa6c904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:52:11.056227 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:11.056838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-08e3d405-d1d4-4d98-93b2-447515624808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:11.058660 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1325/2657] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:52:10 2022] DELETE /identity/v3/projects/e61255d9a2f04207b2e34b68efa6c904 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:11.063826 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e636-2d7a-49a4-aa95-cfc174b8683e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.064118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e636-2d7a-49a4-aa95-cfc174b8683e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.064520 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e636-2d7a-49a4-aa95-cfc174b8683e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.083534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0a01e636-2d7a-49a4-aa95-cfc174b8683e None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:11.084748 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0a01e636-2d7a-49a4-aa95-cfc174b8683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:11.094576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a6b8b-8ecb-4e62-83bb-5c36a1390771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.095531 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6b8b-8ecb-4e62-83bb-5c36a1390771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.095894 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6b8b-8ecb-4e62-83bb-5c36a1390771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.096263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6b8b-8ecb-4e62-83bb-5c36a1390771 admin admin] PATH_INFO: `/v3/projects/d17edf9c20f0426db1c8a1dbc420ed6c/users/bc0ab664f7184cfd9bd6170e92512c48/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.111605 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6b8b-8ecb-4e62-83bb-5c36a1390771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17edf9c20f0426db1c8a1dbc420ed6c, user_id=bc0ab664f7184cfd9bd6170e92512c48, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.114296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6b8b-8ecb-4e62-83bb-5c36a1390771 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.127215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1333/2658] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:52:11 2022] PUT /identity/v3/projects/d17edf9c20f0426db1c8a1dbc420ed6c/users/bc0ab664f7184cfd9bd6170e92512c48/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:52:11.133564 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.133852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.134131 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.145220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1326/2659] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:11.152971 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:11.153130 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.153253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.154160 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.155196 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:11.172057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:11.173289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:11.215027 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1334/2660] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:11.226722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.228116 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:11.233504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1327/2661] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:11.237697 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.246270 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6706ba-e146-4e1c-9338-3757ef81bb94 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.250833 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.251463 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.251741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.252007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.252789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:52:11.253090 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.254905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.267510 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1328/2662] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:52:11.272004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.282741 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7013-8778-4f5b-b891-8e419469e198 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.290460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.291123 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.291403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.291679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.292299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7e715-5abc-4f7b-8d53-30002cb4d747 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.292441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.294351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.296261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.296898 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.297177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.297476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.307666 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.309237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.322979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1329/2664] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:52:11 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:52:11.328212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.328524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1335/2664] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:52:11.393113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fe7b-64af-4b4c-95e9-ef29acc31ddf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.398803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.399456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.399869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.400152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.400863 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.402789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.432072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1330/2665] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:52:11 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:52:11.433369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.444712 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe18f61-63f9-4c5b-a870-0065ffc60793 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.449907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.451204 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.451517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.451763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.452538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.454124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.464642 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1336/2666] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:52:11.468936 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.479942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdea9a5-2fea-4258-be46-6d4599054c4b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.485044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8960c-07ab-414f-927b-72265a9d0330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.485779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8960c-07ab-414f-927b-72265a9d0330 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.486070 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8960c-07ab-414f-927b-72265a9d0330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.486349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8960c-07ab-414f-927b-72265a9d0330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.487096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8960c-07ab-414f-927b-72265a9d0330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.488769 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8960c-07ab-414f-927b-72265a9d0330 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.499558 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1331/2667] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:52:11.504032 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.518733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a5dd9-3ded-4def-a959-3b53dcd1538a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.525139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.526058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.526342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.526627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] PATH_INFO: `/v3/projects/d7bad89fe50547a0928dedf414f9e5f4/users/0b6775f1de844e39a9aa58c9f981d16c/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.542904 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bad89fe50547a0928dedf414f9e5f4, user_id=0b6775f1de844e39a9aa58c9f981d16c, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.545829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.561295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1337/2668] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:52:11 2022] PUT /identity/v3/projects/d7bad89fe50547a0928dedf414f9e5f4/users/0b6775f1de844e39a9aa58c9f981d16c/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:52:11.572809 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.573195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.573478 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.603258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] MFA Rules not processed for user `0b6775f1de844e39a9aa58c9f981d16c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:11.604639 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:11.684377 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1332/2669] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:11.696230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:11.698065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:11.759074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be439799-e221-4540-b102-e74c4f4935f5 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:11.763516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279707c-8f56-4dfc-8711-efe9176b737a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:11.764425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279707c-8f56-4dfc-8711-efe9176b737a service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:11.764638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279707c-8f56-4dfc-8711-efe9176b737a service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:11.764919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279707c-8f56-4dfc-8711-efe9176b737a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:11.774153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279707c-8f56-4dfc-8711-efe9176b737a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:11.775821 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279707c-8f56-4dfc-8711-efe9176b737a service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:11.787819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1338/2670] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:15.065313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:15.154838 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510789f-0894-4da3-b770-cbb78367c05f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:15.160664 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:15.161617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:15.161965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:15.162306 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] PATH_INFO: `/v3/users/0b6775f1de844e39a9aa58c9f981d16c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:15.163709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6775f1de844e39a9aa58c9f981d16c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:15.166154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:15.242614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6775f1de844e39a9aa58c9f981d16c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:15.254158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6775f1de844e39a9aa58c9f981d16c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:15.264782 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6775f1de844e39a9aa58c9f981d16c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:15.265609 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6775f1de844e39a9aa58c9f981d16c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:15.280719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6775f1de844e39a9aa58c9f981d16c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:15.291879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1333/2671] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:52:15 2022] DELETE /identity/v3/users/0b6775f1de844e39a9aa58c9f981d16c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:15.293738 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:15.294683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:15.295401 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:15.316862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:15.318944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:15.405091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1339/2672] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:15.416354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:15.418168 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:15.490146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015a4db-0b25-4070-8908-3652e6850e97 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:15.498621 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf924f38-633e-4389-ac52-c0fad52372fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:15.499688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf924f38-633e-4389-ac52-c0fad52372fe service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:15.500057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf924f38-633e-4389-ac52-c0fad52372fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:15.500448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf924f38-633e-4389-ac52-c0fad52372fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:15.510389 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf924f38-633e-4389-ac52-c0fad52372fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:15.511840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf924f38-633e-4389-ac52-c0fad52372fe service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:15.524693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1334/2673] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:15.898224 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:15.902227 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:15.923344 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3cecc-2c9e-4b9d-b90d-2012bfa4f2ab None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:15.937880 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:15.939517 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:15.940083 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:15.940636 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] PATH_INFO: `/v3/projects/d7bad89fe50547a0928dedf414f9e5f4` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:15.942282 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bad89fe50547a0928dedf414f9e5f4)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:15.944572 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.009669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bad89fe50547a0928dedf414f9e5f4', 'request_id': 'req-a207ecc7-784b-4c7b-9112-79264a4c8c6f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:16.026326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bad89fe50547a0928dedf414f9e5f4', 'request_id': 'req-a207ecc7-784b-4c7b-9112-79264a4c8c6f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:16.033704 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bad89fe50547a0928dedf414f9e5f4', 'request_id': 'req-a207ecc7-784b-4c7b-9112-79264a4c8c6f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:16.047756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] The token cache is being invalidate because project d7bad89fe50547a0928dedf414f9e5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:52:16.048056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:16.048761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-a207ecc7-784b-4c7b-9112-79264a4c8c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:52:16.051487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1340/2674] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:52:15 2022] DELETE /identity/v3/projects/d7bad89fe50547a0928dedf414f9e5f4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:16.057727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.058034 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.058290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.081427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:16.082762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:16.166802 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1335/2675] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:16.174988 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.175234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.175516 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.200009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:16.202123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:16.215872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1341/2676] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:16.220862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.235167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680acca2-9b91-4064-8c36-e75b3c540dac None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.241894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.242551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.242827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.246046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.246046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:52:16.246046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.246046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.269313 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1336/2677] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:52:16.273288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.287054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7de48-5612-41d2-87b4-90f8a0ab4678 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.292153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837600-99e6-4b3c-ace0-c41c1ea51270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.292985 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837600-99e6-4b3c-ace0-c41c1ea51270 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.293481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837600-99e6-4b3c-ace0-c41c1ea51270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.294455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837600-99e6-4b3c-ace0-c41c1ea51270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.295753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d837600-99e6-4b3c-ace0-c41c1ea51270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.297887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d837600-99e6-4b3c-ace0-c41c1ea51270 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.329407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1342/2678] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:52:16 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:52:16.333627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.352651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.353283 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.353803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.354574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f140f94-c8ad-466d-949e-0e88a4ae1131 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.390109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] MFA Rules not processed for user `bc0ab664f7184cfd9bd6170e92512c48`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:16.391453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:16.444911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.445407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.445708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.445975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.446670 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.452617 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.479181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1337/2679] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:52:16 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:52:16.483469 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.498107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1343/2680] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:16.498324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910447d-1b28-4999-a196-809844d6f6fa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.502939 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.503612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.504156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.504459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.505214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.505797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.506799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.507285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:16.520406 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1338/2681] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:52:16.522274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.532358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98f705-5983-4d54-81e5-ef9b1100ca41 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.537484 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.538142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.539058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.539342 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.540248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.541924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.554746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1339/2682] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:52:16.558988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.572671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a383e-705a-4f97-b00c-f1e73b6303d8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.585438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6927-5e76-4bd9-a5b4-f10cab4d81e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.587494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6927-5e76-4bd9-a5b4-f10cab4d81e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.588198 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6927-5e76-4bd9-a5b4-f10cab4d81e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.588539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6927-5e76-4bd9-a5b4-f10cab4d81e0 admin admin] PATH_INFO: `/v3/projects/22f9cad467614c16acb967c11da20d7f/users/9c6f9b63000c47628980c8b435f12043/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.608815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b24c41-4e9a-4e83-ac98-e5461df61a00 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.621304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f6927-5e76-4bd9-a5b4-f10cab4d81e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f9cad467614c16acb967c11da20d7f, user_id=9c6f9b63000c47628980c8b435f12043, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.624533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.625199 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.625326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f6927-5e76-4bd9-a5b4-f10cab4d81e0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.625502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.625750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.634648 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.636290 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.644229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1340/2683] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:52:16 2022] PUT /identity/v3/projects/22f9cad467614c16acb967c11da20d7f/users/9c6f9b63000c47628980c8b435f12043/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:16.645842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1344/2684] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:16.650498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e959d-4ecf-485b-bc2b-7b1e388cef4c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.650789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e959d-4ecf-485b-bc2b-7b1e388cef4c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.651066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e959d-4ecf-485b-bc2b-7b1e388cef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.672616 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-038e959d-4ecf-485b-bc2b-7b1e388cef4c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:16.674151 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-038e959d-4ecf-485b-bc2b-7b1e388cef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:16.753043 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1341/2685] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:16.764491 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:16.765909 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:16.830748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22898132-e3bc-4e0e-a5f7-1baf50b44a3c service nova] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:16.882228 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5c162-40d8-4bff-9a34-fcf33b448392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:16.883189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5c162-40d8-4bff-9a34-fcf33b448392 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:16.883470 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5c162-40d8-4bff-9a34-fcf33b448392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:16.883745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5c162-40d8-4bff-9a34-fcf33b448392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:16.895305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5c162-40d8-4bff-9a34-fcf33b448392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:16.896803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5c162-40d8-4bff-9a34-fcf33b448392 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:16.908336 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1345/2686] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:19.426032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:19.427713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:19.428225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:19.461832 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] MFA Rules not processed for user `bc0ab664f7184cfd9bd6170e92512c48`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:19.463815 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:19.601521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1342/2687] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:20.217688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f89f1f-0d5a-4e5a-9f9f-4cc868f5598c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:20.218590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f89f1f-0d5a-4e5a-9f9f-4cc868f5598c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:20.219033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f89f1f-0d5a-4e5a-9f9f-4cc868f5598c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:20.240913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-93f89f1f-0d5a-4e5a-9f9f-4cc868f5598c None None] MFA Rules not processed for user `adbaa9b8b7e2447da5eef8f32aff22a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:20.245926 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-93f89f1f-0d5a-4e5a-9f9f-4cc868f5598c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:20.322246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1346/2688] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:21.248248 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:21.264092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ed602-2fa6-4ab3-a182-0c27a85ba299 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:21.269361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:21.270482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:21.271195 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:21.271695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:21.275001 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:21.293653 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:21.295671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:21.316581 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1343/2689] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:24.375683 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:24.376447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:24.376447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:24.428181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] MFA Rules not processed for user `9c6f9b63000c47628980c8b435f12043`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:24.429759 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:24.581815 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1347/2690] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:24.587144 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:24.605422 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153a06-8e2f-4c18-b32f-135c9d4ddfa5 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:24.623272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:24.624843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:24.625571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:24.626944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:24.629393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:24.633357 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:24.699536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1344/2691] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:52:24 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:52:24.700569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:24.721477 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c15c6-8fb0-4db0-8e23-39c99dd4ab52 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:24.801656 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:24.802436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:24.802839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:24.803181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:24.804099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:24.806045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:24.834453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1348/2692] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:52:24 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:52:24.837878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:24.858690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e6b78-368b-4723-9d31-f2c687eec487 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:24.864453 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:24.866701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:24.868513 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:24.868988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:24.871585 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:24.875445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:24.892502 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1345/2693] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:52:24.896316 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:24.909939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1fd46-c5e1-444e-a3cc-8f2ef1d460c0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:24.914843 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:24.916121 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:24.916510 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:24.916864 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] PATH_INFO: `/v3/projects/908a3d1181774f4d9d8f347d64bd4771/users/f56e3f1b655240fcb1cbbd654d59d971/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:24.949710 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908a3d1181774f4d9d8f347d64bd4771, user_id=f56e3f1b655240fcb1cbbd654d59d971, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:24.954813 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:24.977836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1349/2694] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:52:24 2022] PUT /identity/v3/projects/908a3d1181774f4d9d8f347d64bd4771/users/f56e3f1b655240fcb1cbbd654d59d971/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:52:24.982345 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:25.000552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b98cd2-4774-48b2-a118-97ddb26d77db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:25.124303 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:25.125118 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:25.125474 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:25.125845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:25.126652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:25.129065 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:25.144687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1346/2695] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:52:25.149942 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:25.165930 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aed155-e92b-4c82-94cc-328e58f370c1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:25.177525 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc183fd-9119-49f1-bcd6-e93c71a75c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:25.178689 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc183fd-9119-49f1-bcd6-e93c71a75c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:25.179694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc183fd-9119-49f1-bcd6-e93c71a75c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:25.179694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc183fd-9119-49f1-bcd6-e93c71a75c70 admin admin] PATH_INFO: `/v3/projects/908a3d1181774f4d9d8f347d64bd4771/users/f56e3f1b655240fcb1cbbd654d59d971/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:25.183222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc183fd-9119-49f1-bcd6-e93c71a75c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908a3d1181774f4d9d8f347d64bd4771, user_id=f56e3f1b655240fcb1cbbd654d59d971, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:25.186785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc183fd-9119-49f1-bcd6-e93c71a75c70 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:25.202229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1350/2696] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:52:25 2022] PUT /identity/v3/projects/908a3d1181774f4d9d8f347d64bd4771/users/f56e3f1b655240fcb1cbbd654d59d971/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:52:29.323603 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:29.485979 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049d352-2e47-4f37-a381-2aa8836ddf55 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:29.571014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:29.572360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:29.572644 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:29.572919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:29.574836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:29.594546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:29.596557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:29.621220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1347/2697] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:33.797887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dcd97-0fbc-4928-8fa0-1198151cae39 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:33.798362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dcd97-0fbc-4928-8fa0-1198151cae39 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:33.798469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dcd97-0fbc-4928-8fa0-1198151cae39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:33.828603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-b80dcd97-0fbc-4928-8fa0-1198151cae39 None None] MFA Rules not processed for user `f56e3f1b655240fcb1cbbd654d59d971`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:33.830124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b80dcd97-0fbc-4928-8fa0-1198151cae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:33.923974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1351/2698] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:33.939569 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:33.942831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:34.019789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6b12-44f1-400e-a434-f37f6021d949 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:34.025574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5621c3ccfb8040748f8adef3dc810c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:34.026439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:34.026876 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:34.027315 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:34.094693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:34.096470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:34.106424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1348/2699] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:34.201460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:34.201783 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:34.202071 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:34.219281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:34.220514 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:34.276280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1352/2700] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:52:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:52:34.324627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:34.326737 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:34.395557 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701ba86-b72e-40ae-84c5-6a7a4e486b41 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:34.400888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e65e-e48f-40d4-a070-72413b7d2ccf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:34.401551 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e65e-e48f-40d4-a070-72413b7d2ccf service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:34.401830 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e65e-e48f-40d4-a070-72413b7d2ccf service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:34.402108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e65e-e48f-40d4-a070-72413b7d2ccf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:34.411286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06e65e-e48f-40d4-a070-72413b7d2ccf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:34.412855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06e65e-e48f-40d4-a070-72413b7d2ccf service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:34.421958 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1349/2701] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:52:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:52:44.246363 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:44.247925 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:44.306044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379395-31eb-47cc-9dd0-6654ce3df471 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:44.310274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:44.310947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:44.311236 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:44.311509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:44.313390 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:44.371295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:44.371879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:44.371879 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:44.377225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:44.378664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061413c-3d23-4f9b-a814-405b706f0d6c service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:44.389090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1353/2702] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:52:44.389286 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] MFA Rules not processed for user `9c6f9b63000c47628980c8b435f12043`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:44.390544 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:44.397634 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1350/2703] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:46.655304 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:46.655809 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:46.655896 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:46.677768 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:52:46.679242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:52:46.693423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1354/2704] 10.222.0.73 () {58 vars in 1009 bytes} [Thu May 12 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:52:46.708125 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:52:46.732386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca631c3-1a66-4f6f-bec6-ebeebcf79fc1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:52:46.743241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:52:46.744150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:52:46.744708 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:52:46.745148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:52:46.755007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:52:46.757112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:52:46.766622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1351/2705] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:52:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:53:14.338948 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:53:14.348632 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a2554-34e0-49cc-9bfa-36592c5e51f8 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:53:14.352124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c245eea-0865-4349-9d8c-983223f89b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:53:14.352775 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c245eea-0865-4349-9d8c-983223f89b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:53:14.353054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c245eea-0865-4349-9d8c-983223f89b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:53:14.353326 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c245eea-0865-4349-9d8c-983223f89b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:53:14.361893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c245eea-0865-4349-9d8c-983223f89b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:53:14.363298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c245eea-0865-4349-9d8c-983223f89b51 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:53:14.372369 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1355/2706] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:53:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:54:06.219411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:06.220842 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:06.294110 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fd3c-5d15-4a8e-8200-f8a192de0c78 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:06.299193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:06.299825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:06.300102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:06.300412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:06.302150 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:06.313794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:06.315361 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:06.325546 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1352/2707] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:54:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:54:28.536331 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c1ba-1fc5-4f48-a4b4-f07afd917123 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:28.536908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c1ba-1fc5-4f48-a4b4-f07afd917123 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:28.536908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953c1ba-1fc5-4f48-a4b4-f07afd917123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:28.553919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-f953c1ba-1fc5-4f48-a4b4-f07afd917123 None None] MFA Rules not processed for user `adbaa9b8b7e2447da5eef8f32aff22a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:54:28.555124 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f953c1ba-1fc5-4f48-a4b4-f07afd917123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:28.607660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1356/2708] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:54:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:54:37.796534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:37.809343 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d228-0ab8-4588-80f5-7026a29d61dc service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:37.814703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:37.815341 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:37.815624 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:37.815892 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:37.817682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:37.827980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:37.829567 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:37.838733 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1353/2709] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:54:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:54:45.366179 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:45.366681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:45.366791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:45.385099 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] MFA Rules not processed for user `3a892d7213414aa589db028d1b7c27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:54:45.386842 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:45.394347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1357/2710] 10.222.0.73 () {58 vars in 967 bytes} [Thu May 12 17:54:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:54:45.438020 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:45.449784 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d518443-81f5-4306-a0f0-bf8f13c3647f service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:45.454726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '972f23cd94bf4051825701d2aaddfe6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:45.455366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:45.455648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:45.455922 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:45.464186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:45.466428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:45.475138 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1354/2711] 10.222.0.73 () {60 vars in 1384 bytes} [Thu May 12 17:54:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 12 17:54:54.782382 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:54.852075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de5931-fe64-4a1b-87fd-785e90d5f054 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:54.857381 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:54.858241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:54.858524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:54.858800 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] PATH_INFO: `/v3/users/9c6f9b63000c47628980c8b435f12043` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:54.859990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6f9b63000c47628980c8b435f12043)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:54.861954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:54.906190 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6f9b63000c47628980c8b435f12043'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:54.918091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6f9b63000c47628980c8b435f12043'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:54.918665 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6f9b63000c47628980c8b435f12043'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:54.919260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6f9b63000c47628980c8b435f12043'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:54.925791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6f9b63000c47628980c8b435f12043'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:54.938496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1358/2712] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:54:54 2022] DELETE /identity/v3/users/9c6f9b63000c47628980c8b435f12043 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:54:55.129998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:55.141887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e31cc-c1fa-4355-8e78-36b65b34293f service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:55.197682 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:55.199327 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:55.199665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:55.199944 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] PATH_INFO: `/v3/projects/22f9cad467614c16acb967c11da20d7f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:55.201171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f9cad467614c16acb967c11da20d7f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:55.203091 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:55.252384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f9cad467614c16acb967c11da20d7f', 'request_id': 'req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.258897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f9cad467614c16acb967c11da20d7f', 'request_id': 'req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.265012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f9cad467614c16acb967c11da20d7f', 'request_id': 'req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.277074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] The token cache is being invalidate because project 22f9cad467614c16acb967c11da20d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:54:55.277359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.278612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.280746 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1355/2713] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:54:55 2022] DELETE /identity/v3/projects/22f9cad467614c16acb967c11da20d7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:54:55.284721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:55.286106 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:55.299343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f92c22-8640-46a8-af1f-d7abce4bb273 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:55.362417 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:55.363380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:55.363798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:55.364079 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] PATH_INFO: `/v3/users/f56e3f1b655240fcb1cbbd654d59d971` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:55.365375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56e3f1b655240fcb1cbbd654d59d971)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:55.367305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:55.414852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56e3f1b655240fcb1cbbd654d59d971'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.427039 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56e3f1b655240fcb1cbbd654d59d971'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.427725 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56e3f1b655240fcb1cbbd654d59d971'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.428365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56e3f1b655240fcb1cbbd654d59d971'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.434981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56e3f1b655240fcb1cbbd654d59d971'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.447241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1359/2714] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:54:55 2022] DELETE /identity/v3/users/f56e3f1b655240fcb1cbbd654d59d971 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:54:55.620923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:55.622344 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:55.634868 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f28ea8-a29f-4e94-96d5-a18853dfdb0d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:55.690771 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:55.691626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:55.691907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:55.692212 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] PATH_INFO: `/v3/projects/908a3d1181774f4d9d8f347d64bd4771` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:55.693418 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908a3d1181774f4d9d8f347d64bd4771)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:55.695308 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:55.740381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908a3d1181774f4d9d8f347d64bd4771', 'request_id': 'req-5d98e8ed-a997-40fa-a907-450799568175'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.747350 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908a3d1181774f4d9d8f347d64bd4771', 'request_id': 'req-5d98e8ed-a997-40fa-a907-450799568175'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.752699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908a3d1181774f4d9d8f347d64bd4771', 'request_id': 'req-5d98e8ed-a997-40fa-a907-450799568175'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.766112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] The token cache is being invalidate because project 908a3d1181774f4d9d8f347d64bd4771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:54:55.766637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.767450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:54:55.769480 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1356/2715] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:54:55 2022] DELETE /identity/v3/projects/908a3d1181774f4d9d8f347d64bd4771 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:54:55.840091 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:55.841664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:55.908826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069d25-da68-44c5-8e0e-147b15995eef admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:55.962550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:55.963221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:55.963502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:55.963779 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:55.966052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:56.038901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:56.040423 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:56.051604 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1360/2716] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:54:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:54:57.225638 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:57.227092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:57.239003 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98e8ed-a997-40fa-a907-450799568175 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:57.242592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:57.243221 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:57.243503 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:57.243777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:57.245521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:57.258442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:57.259860 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:57.270970 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1357/2717] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:54:59.849915 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:54:59.859486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f27028-97c8-4c05-a702-0e4095fa65c5 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:54:59.863080 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:54:59.863717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:54:59.863998 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:54:59.864298 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:54:59.866105 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:54:59.875782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:54:59.877239 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270c82a-c3c7-4eba-8fde-8f1bbbd9c39d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:54:59.885597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1361/2718] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:54:59.994258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:55:00.003317 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522cc95-23e8-46a2-b6ba-0012776b255a service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:55:00.007426 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:55:00.008106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:55:00.008424 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:55:00.008725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:55:00.011249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:00.025612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:55:00.027107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:55:00.037986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1358/2719] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:55:28.961651 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:55:28.962185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:55:28.962305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:55:28.980716 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] MFA Rules not processed for user `bc0ab664f7184cfd9bd6170e92512c48`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:55:28.981927 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:29.037162 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1362/2720] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:55:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:55:29.673612 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:55:29.675054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:29.742709 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0c1a0-63d2-4aab-ad6f-055ad8bfaa0d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:55:29.747552 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:55:29.748225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:55:29.748508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:55:29.748780 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:55:29.750515 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:29.762326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:55:29.763854 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:55:29.773443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1359/2721] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:55:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:55:31.600438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:55:31.601992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:31.616550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ab7e-3b51-4a0e-8275-955dfbc5dcc5 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:55:31.620625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:55:31.621285 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:55:31.621568 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:55:31.621852 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:55:31.623711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:31.634033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:55:31.636023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:55:31.645123 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1363/2722] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:55:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:55:32.297194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:55:32.299823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:32.319934 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810950b-3e87-4577-ad98-22925158c33f service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:55:32.327577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:55:32.328542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:55:32.329036 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:55:32.329528 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:55:32.332056 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:55:32.353164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:55:32.355543 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:55:32.367497 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1360/2723] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:55:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:56:41.999358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:42.001441 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:42.082640 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e3854-c8fe-4d02-9a0a-26ad2d3c2123 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:42.087934 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:42.088879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:42.089165 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:42.089498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] PATH_INFO: `/v3/users/7cee74fc982d4aa39f56c6b40de259cc` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:42.090846 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cee74fc982d4aa39f56c6b40de259cc)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:42.092826 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:42.140230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cee74fc982d4aa39f56c6b40de259cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.152247 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cee74fc982d4aa39f56c6b40de259cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.152945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cee74fc982d4aa39f56c6b40de259cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.153585 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cee74fc982d4aa39f56c6b40de259cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.160319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cee74fc982d4aa39f56c6b40de259cc'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.174066 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1364/2724] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:56:41 2022] DELETE /identity/v3/users/7cee74fc982d4aa39f56c6b40de259cc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:42.346428 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:42.357665 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec2ee1-fb81-48db-a4c8-450c0c45cc76 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:42.424799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:42.425565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:42.425843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:42.426114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] PATH_INFO: `/v3/projects/583159e196b2438698885193ca03ef4d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:42.427279 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] RBAC: Authorizing `identity:delete_project(project_id=583159e196b2438698885193ca03ef4d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:42.429202 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:42.477225 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583159e196b2438698885193ca03ef4d', 'request_id': 'req-e9b1f566-280c-4454-a535-ac5e56c72def'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.483295 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583159e196b2438698885193ca03ef4d', 'request_id': 'req-e9b1f566-280c-4454-a535-ac5e56c72def'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.488673 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583159e196b2438698885193ca03ef4d', 'request_id': 'req-e9b1f566-280c-4454-a535-ac5e56c72def'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.501083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] The token cache is being invalidate because project 583159e196b2438698885193ca03ef4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:56:42.501366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.501987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.503518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1361/2725] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:56:42 2022] DELETE /identity/v3/projects/583159e196b2438698885193ca03ef4d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:42.508411 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:42.509918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:42.524020 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aea8f-2e7f-457e-b8c9-9c712bf4a6f1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:42.581416 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:42.582294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:42.582569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:42.582837 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] PATH_INFO: `/v3/users/8ccbd90986a34e6ba1e74df6c56aa386` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:42.584060 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccbd90986a34e6ba1e74df6c56aa386)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:42.586067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:42.636313 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccbd90986a34e6ba1e74df6c56aa386'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.648056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccbd90986a34e6ba1e74df6c56aa386'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.648858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccbd90986a34e6ba1e74df6c56aa386'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.649452 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccbd90986a34e6ba1e74df6c56aa386'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.656715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccbd90986a34e6ba1e74df6c56aa386'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:42.670153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1365/2726] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:56:42 2022] DELETE /identity/v3/users/8ccbd90986a34e6ba1e74df6c56aa386 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:42.861929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:42.863442 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:42.877843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1f566-280c-4454-a535-ac5e56c72def admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:42.942159 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:42.943016 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:42.943291 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:42.943559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] PATH_INFO: `/v3/projects/72c4c25ece984355b0f8eb63e27c99b8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:42.944767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c4c25ece984355b0f8eb63e27c99b8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:42.946775 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:43.013137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c4c25ece984355b0f8eb63e27c99b8', 'request_id': 'req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:43.020978 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c4c25ece984355b0f8eb63e27c99b8', 'request_id': 'req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:43.027191 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c4c25ece984355b0f8eb63e27c99b8', 'request_id': 'req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:43.045929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] The token cache is being invalidate because project 72c4c25ece984355b0f8eb63e27c99b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:56:43.046444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:43.047473 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:43.050228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1362/2727] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:56:42 2022] DELETE /identity/v3/projects/72c4c25ece984355b0f8eb63e27c99b8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:44.763791 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:44.765235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:44.822341 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfc2e6-3c99-4add-9cb3-3b2020aee82d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:44.876410 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:44.877438 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:44.877744 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:44.878023 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:44.880657 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:44.968551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:44.970210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:44.983044 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1366/2728] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:56:51.778470 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:51.780153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:51.793487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a350ef-919d-4f01-ae0e-1c4aaeb4da6c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:51.798085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:51.798950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:51.799231 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:51.799504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] PATH_INFO: `/v3/users/adbaa9b8b7e2447da5eef8f32aff22a1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:51.800749 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adbaa9b8b7e2447da5eef8f32aff22a1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:51.802689 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:51.852700 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adbaa9b8b7e2447da5eef8f32aff22a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:51.859571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adbaa9b8b7e2447da5eef8f32aff22a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:51.867656 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adbaa9b8b7e2447da5eef8f32aff22a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:51.868666 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adbaa9b8b7e2447da5eef8f32aff22a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:51.880046 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adbaa9b8b7e2447da5eef8f32aff22a1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:51.887066 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1363/2729] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:56:51 2022] DELETE /identity/v3/users/adbaa9b8b7e2447da5eef8f32aff22a1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:52.075010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:52.085637 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544ddc1-18af-4c0e-97e1-72ef4ea7c31f service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:52.146919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:52.147802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:52.148113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:52.148804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] PATH_INFO: `/v3/projects/a4d7ebeafe8140b3b8332059258a83eb` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:52.149977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d7ebeafe8140b3b8332059258a83eb)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:52.151884 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:52.197541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d7ebeafe8140b3b8332059258a83eb', 'request_id': 'req-86f94298-7439-4c8e-a0fb-f6883d3cc162'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.206092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d7ebeafe8140b3b8332059258a83eb', 'request_id': 'req-86f94298-7439-4c8e-a0fb-f6883d3cc162'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.212427 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d7ebeafe8140b3b8332059258a83eb', 'request_id': 'req-86f94298-7439-4c8e-a0fb-f6883d3cc162'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.223028 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] The token cache is being invalidate because project a4d7ebeafe8140b3b8332059258a83eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:56:52.223312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.223932 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.225460 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1367/2730] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:56:52 2022] DELETE /identity/v3/projects/a4d7ebeafe8140b3b8332059258a83eb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:52.229572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:52.231157 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:52.244232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f34dc-c06d-4300-abd3-03785e086d11 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:52.300247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:52.301054 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:52.301335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:52.301606 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] PATH_INFO: `/v3/users/22beb108fd7d4479b827cd11e2383774` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:52.302787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22beb108fd7d4479b827cd11e2383774)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:52.305245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:52.349289 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22beb108fd7d4479b827cd11e2383774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.356136 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22beb108fd7d4479b827cd11e2383774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.363126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22beb108fd7d4479b827cd11e2383774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.363756 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22beb108fd7d4479b827cd11e2383774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.376049 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22beb108fd7d4479b827cd11e2383774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.381993 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1364/2731] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:56:52 2022] DELETE /identity/v3/users/22beb108fd7d4479b827cd11e2383774 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:52.552807 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:52.554954 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:52.569053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94298-7439-4c8e-a0fb-f6883d3cc162 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:52.628619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:52.629521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:52.629804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:52.630109 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] PATH_INFO: `/v3/projects/d7c90277db3d4f89abd17dc952d72400` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:52.631645 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c90277db3d4f89abd17dc952d72400)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:52.634502 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:52.683597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c90277db3d4f89abd17dc952d72400', 'request_id': 'req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.690836 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c90277db3d4f89abd17dc952d72400', 'request_id': 'req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.697356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c90277db3d4f89abd17dc952d72400', 'request_id': 'req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.708254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] The token cache is being invalidate because project d7c90277db3d4f89abd17dc952d72400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:56:52.708543 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.709198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:52.711739 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1368/2732] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:56:52 2022] DELETE /identity/v3/projects/d7c90277db3d4f89abd17dc952d72400 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:58.770509 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:58.772148 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:58.787564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15d45-3d8a-4938-b3e6-adcf455ed969 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:58.844052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:58.845000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:58.845232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:58.845504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] PATH_INFO: `/v3/users/bc0ab664f7184cfd9bd6170e92512c48` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:58.846766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc0ab664f7184cfd9bd6170e92512c48)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:58.848818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:58.905719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc0ab664f7184cfd9bd6170e92512c48'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:58.912917 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc0ab664f7184cfd9bd6170e92512c48'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:58.921732 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc0ab664f7184cfd9bd6170e92512c48'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:58.922364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc0ab664f7184cfd9bd6170e92512c48'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:58.935850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c1a7e51-7344-4fd2-b72f-55d9896fd6fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc0ab664f7184cfd9bd6170e92512c48'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:58.942354 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1365/2733] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:56:58 2022] DELETE /identity/v3/users/bc0ab664f7184cfd9bd6170e92512c48 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:59.117592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.119181 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:59.132519 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ee4dd-ec71-4ff5-ad10-a77320a4ff8a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.191361 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.192249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.192532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.192835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] PATH_INFO: `/v3/projects/d17edf9c20f0426db1c8a1dbc420ed6c` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.194248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17edf9c20f0426db1c8a1dbc420ed6c)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.196260 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.245293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17edf9c20f0426db1c8a1dbc420ed6c', 'request_id': 'req-b85c8d8b-c169-4b00-8e6c-c726287b733f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:59.252207 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17edf9c20f0426db1c8a1dbc420ed6c', 'request_id': 'req-b85c8d8b-c169-4b00-8e6c-c726287b733f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:59.267334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17edf9c20f0426db1c8a1dbc420ed6c', 'request_id': 'req-b85c8d8b-c169-4b00-8e6c-c726287b733f'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:59.284731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] The token cache is being invalidate because project d17edf9c20f0426db1c8a1dbc420ed6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:56:59.285193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:59.285715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b85c8d8b-c169-4b00-8e6c-c726287b733f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:56:59.287569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1369/2734] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:56:59 2022] DELETE /identity/v3/projects/d17edf9c20f0426db1c8a1dbc420ed6c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:59.294232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.294232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.294699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.314441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:56:59.315645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:59.378441 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1366/2735] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:56:59.390033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.390342 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.390628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.410037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:56:59.411233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:59.419981 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1370/2736] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:56:59.424294 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.433703 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf1fa1-865a-4644-a5d0-15853207b11c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.438190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.438823 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.439129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.439398 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.440115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:56:59.440443 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.442007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.453699 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1367/2737] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:56:59.458394 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.468730 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20100cb8-9046-4f28-982e-40c05f03c341 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.473375 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.474017 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.474319 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.474614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.475295 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.477288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.502235 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1371/2738] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:56:59 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:56:59.506723 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.517556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd03831-3e6d-4977-9bb1-de3ac7a7ca67 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.573755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.574401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.574687 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.574977 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.575679 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.577831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.600845 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1368/2739] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:56:59 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:56:59.605072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.619591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ee6f-1ec8-4a59-bd0a-b006c925135e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.625586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.626261 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.626549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.626827 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.627559 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.629188 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.641078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1372/2740] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:56:59.645456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.656099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a04bd-8582-475f-9db2-4700b7e0c970 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.660750 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ed247-1b0d-4040-9b7d-1826f962d863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.661410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed247-1b0d-4040-9b7d-1826f962d863 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.661695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed247-1b0d-4040-9b7d-1826f962d863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.661971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed247-1b0d-4040-9b7d-1826f962d863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.662695 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ed247-1b0d-4040-9b7d-1826f962d863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.664499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ed247-1b0d-4040-9b7d-1826f962d863 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.677596 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1369/2741] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:56:59.681741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.691413 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af881-8df3-405b-aeb1-129d2bac3bda admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.696629 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.697499 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.697780 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.698056 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] PATH_INFO: `/v3/projects/1e5dcc1fb7244096909161967f9a9d74/users/d36b96cf14cb4a5d9e75f14f237082d5/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.712672 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5dcc1fb7244096909161967f9a9d74, user_id=d36b96cf14cb4a5d9e75f14f237082d5, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.715288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.727877 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1373/2742] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:56:59 2022] PUT /identity/v3/projects/1e5dcc1fb7244096909161967f9a9d74/users/d36b96cf14cb4a5d9e75f14f237082d5/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:56:59.737924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.738214 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.738491 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.766432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] MFA Rules not processed for user `d36b96cf14cb4a5d9e75f14f237082d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:56:59.767641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:59.832457 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1370/2743] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:56:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:56:59.845296 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:56:59.847345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:56:59.959912 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97a82-2025-40fe-91c6-9f5d796297fe admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:56:59.966435 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f671fda-6dc9-414c-8861-042bcc98386f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:56:59.967097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f671fda-6dc9-414c-8861-042bcc98386f service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:56:59.967386 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f671fda-6dc9-414c-8861-042bcc98386f service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:56:59.967663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f671fda-6dc9-414c-8861-042bcc98386f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:56:59.978643 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f671fda-6dc9-414c-8861-042bcc98386f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:56:59.980254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f671fda-6dc9-414c-8861-042bcc98386f service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:56:59.992658 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1374/2744] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:56:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:07.294996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:07.397508 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309910ff-5fd7-4cd8-b98a-2f701118a5d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:07.404450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:07.405328 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:07.405601 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:07.405878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] PATH_INFO: `/v3/users/d36b96cf14cb4a5d9e75f14f237082d5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:07.407172 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36b96cf14cb4a5d9e75f14f237082d5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:07.409102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:07.457519 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36b96cf14cb4a5d9e75f14f237082d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:07.466274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36b96cf14cb4a5d9e75f14f237082d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:07.477238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36b96cf14cb4a5d9e75f14f237082d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:07.478852 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36b96cf14cb4a5d9e75f14f237082d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:07.492982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36b96cf14cb4a5d9e75f14f237082d5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:07.500427 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1371/2745] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:07 2022] DELETE /identity/v3/users/d36b96cf14cb4a5d9e75f14f237082d5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:07.506670 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:07.507333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:07.507333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:07.550593 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:07.551817 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:07.619823 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1375/2746] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:07.633383 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:07.634911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:07.739881 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca465e-1704-49d9-8597-5ea0cb9c2125 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:07.744813 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bcca6-2d79-4e6a-95fb-95b3c1964c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:07.745808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcca6-2d79-4e6a-95fb-95b3c1964c7c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:07.746304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcca6-2d79-4e6a-95fb-95b3c1964c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:07.746799 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcca6-2d79-4e6a-95fb-95b3c1964c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:07.761961 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcca6-2d79-4e6a-95fb-95b3c1964c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:07.764626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcca6-2d79-4e6a-95fb-95b3c1964c7c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:07.775783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1372/2747] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:08.071718 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.073380 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:08.086987 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89818a9e-2290-42ab-83fe-5562c3aa0b37 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.093534 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.095368 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.095761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.095939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] PATH_INFO: `/v3/projects/1e5dcc1fb7244096909161967f9a9d74` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.097653 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5dcc1fb7244096909161967f9a9d74)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.099905 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.158305 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5dcc1fb7244096909161967f9a9d74', 'request_id': 'req-42ef97c9-b435-42c4-b659-2d760a59ef5b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:08.166875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5dcc1fb7244096909161967f9a9d74', 'request_id': 'req-42ef97c9-b435-42c4-b659-2d760a59ef5b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:08.174922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5dcc1fb7244096909161967f9a9d74', 'request_id': 'req-42ef97c9-b435-42c4-b659-2d760a59ef5b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:08.188087 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] The token cache is being invalidate because project 1e5dcc1fb7244096909161967f9a9d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:08.188405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:08.189067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-42ef97c9-b435-42c4-b659-2d760a59ef5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:08.190626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1376/2748] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:08 2022] DELETE /identity/v3/projects/1e5dcc1fb7244096909161967f9a9d74 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:08.197844 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.198107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.198488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.250115 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:08.251572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:08.329067 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1373/2749] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:08.337404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.337910 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.338321 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.369614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:08.370908 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:08.380459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1377/2750] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:08.385930 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.398875 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6f60-5c91-42bc-9c51-9c41cfcdd00f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.405870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.406811 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.407098 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.407416 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.408284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:57:08.408591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.410190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.425108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1374/2751] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:57:08.430731 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.443732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea3646-4bc2-4755-add2-3753d1b3b349 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.451661 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.452677 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.453115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.453445 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.454156 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.456125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.487785 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1378/2752] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:08 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:08.493681 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.504449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee75b8-e00c-46c0-b727-c65237f59dd7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.568234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.569060 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.569349 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.569984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.571093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.573048 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.596277 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1375/2753] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:08 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:57:08.601688 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.615986 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22591a67-9570-4ccf-8fdc-4db447ed5445 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.623132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.623849 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.624189 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.624550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.625347 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.627067 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.646562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1379/2754] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:08.652848 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.668514 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713df2ed-ac68-4896-92b9-f4b2e50940df admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.674608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f80bbf-1b1b-4abc-b8eb-976fc4ef1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.675482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80bbf-1b1b-4abc-b8eb-976fc4ef1763 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.675767 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80bbf-1b1b-4abc-b8eb-976fc4ef1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.676111 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f80bbf-1b1b-4abc-b8eb-976fc4ef1763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.676856 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80bbf-1b1b-4abc-b8eb-976fc4ef1763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.678464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f80bbf-1b1b-4abc-b8eb-976fc4ef1763 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.693561 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1376/2755] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:08.699241 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.713511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d070b3c-bbbb-4d69-95ef-54f2036db427 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.721019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.722049 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.722325 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.722599 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] PATH_INFO: `/v3/projects/51a94e5b3e8f4870af4088d6cde36351/users/35f207aa21c94d93aafe89c38f7d7223/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.740302 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a94e5b3e8f4870af4088d6cde36351, user_id=35f207aa21c94d93aafe89c38f7d7223, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.744151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.758469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1380/2756] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:08 2022] PUT /identity/v3/projects/51a94e5b3e8f4870af4088d6cde36351/users/35f207aa21c94d93aafe89c38f7d7223/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:57:08.770467 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.770819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.771131 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.800572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] MFA Rules not processed for user `35f207aa21c94d93aafe89c38f7d7223`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:08.801790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:08.876836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1377/2757] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:57:08.888406 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:08.890180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:08.968556 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93840-61b9-4277-94a2-92f14fa7d81b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:08.974234 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15d321-23ea-4de7-b5d8-2dde499bbc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62c3b98740d14ab49d91783086150e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:08.974879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15d321-23ea-4de7-b5d8-2dde499bbc7f service nova] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:08.975158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15d321-23ea-4de7-b5d8-2dde499bbc7f service nova] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:08.975431 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15d321-23ea-4de7-b5d8-2dde499bbc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:08.984288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15d321-23ea-4de7-b5d8-2dde499bbc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:08.985850 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15d321-23ea-4de7-b5d8-2dde499bbc7f service nova] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:08.995249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1381/2758] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:09.217190 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:09.283950 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3e7c8-e574-46a1-be69-2996574b0544 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:09.288951 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:09.289710 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:09.289982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:09.290247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] PATH_INFO: `/v3/users/35f207aa21c94d93aafe89c38f7d7223` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:09.291417 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f207aa21c94d93aafe89c38f7d7223)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:09.293384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:09.350258 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f207aa21c94d93aafe89c38f7d7223'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.357727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f207aa21c94d93aafe89c38f7d7223'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.369249 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f207aa21c94d93aafe89c38f7d7223'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.370181 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f207aa21c94d93aafe89c38f7d7223'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.384747 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f207aa21c94d93aafe89c38f7d7223'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.391578 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1378/2759] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:09 2022] DELETE /identity/v3/users/35f207aa21c94d93aafe89c38f7d7223 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:09.396715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:09.396715 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:09.398237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:09.415265 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:09.416532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:09.480114 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1382/2760] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:09.490565 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:09.492693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:09.555534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96618ec4-66a2-48a8-9545-dc018c7fd88a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:09.559390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89acd2e-dc59-4a27-9855-bc1942499511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:09.560042 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89acd2e-dc59-4a27-9855-bc1942499511 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:09.560360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89acd2e-dc59-4a27-9855-bc1942499511 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:09.560640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89acd2e-dc59-4a27-9855-bc1942499511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:09.573107 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89acd2e-dc59-4a27-9855-bc1942499511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:09.574787 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89acd2e-dc59-4a27-9855-bc1942499511 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:09.587292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1379/2761] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:09.895887 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:09.897561 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:09.911216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b00d6-328b-4d24-a5dc-15540522ddd0 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:09.916309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:09.917967 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:09.918264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:09.918565 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] PATH_INFO: `/v3/projects/51a94e5b3e8f4870af4088d6cde36351` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:09.921907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a94e5b3e8f4870af4088d6cde36351)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:09.921907 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:09.975459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a94e5b3e8f4870af4088d6cde36351', 'request_id': 'req-5d14a3bf-eb8b-4623-82b6-0fa67c945208'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.984761 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a94e5b3e8f4870af4088d6cde36351', 'request_id': 'req-5d14a3bf-eb8b-4623-82b6-0fa67c945208'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:09.993495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a94e5b3e8f4870af4088d6cde36351', 'request_id': 'req-5d14a3bf-eb8b-4623-82b6-0fa67c945208'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:10.012233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] The token cache is being invalidate because project 51a94e5b3e8f4870af4088d6cde36351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80296) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:10.012578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:10.013240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-5d14a3bf-eb8b-4623-82b6-0fa67c945208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:10.014890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1383/2762] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:09 2022] DELETE /identity/v3/projects/51a94e5b3e8f4870af4088d6cde36351 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:10.020117 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.020538 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.020819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.040488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:10.041728 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:10.103727 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1380/2763] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:57:10.111833 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.112097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.112475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.132839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:10.134016 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:10.142096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1384/2764] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:10.146433 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.157292 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0e0aa-4166-4c23-807e-04d345c64584 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.161850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.162505 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.162783 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.163052 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.163840 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:57:10.164147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.165764 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.177831 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1381/2765] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:57:10.182443 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.192078 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aff4ca-d747-42a6-bf31-947b72c4fcfd None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.198288 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.198951 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.199229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.199503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.200230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.202130 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.232554 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1385/2766] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:10.237928 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.248674 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdeaf7-77a8-4514-8d5f-245923666475 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.311147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.311853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.312092 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.312425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.313129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.315023 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.348461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1382/2767] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:10.353628 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.364425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86552436-908a-4094-b028-7aafdf2761ac admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.369963 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.370638 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.370919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.371193 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.371977 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.373603 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.386664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1386/2768] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:10.390919 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.401512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c317ab5-0f68-4503-92d7-96989a86b19c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.406407 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3b5a3-dfca-49c3-8cf7-b3e059d6d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.407073 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3b5a3-dfca-49c3-8cf7-b3e059d6d9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.407358 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3b5a3-dfca-49c3-8cf7-b3e059d6d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.407667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3b5a3-dfca-49c3-8cf7-b3e059d6d9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.408444 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3b5a3-dfca-49c3-8cf7-b3e059d6d9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.410072 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3b5a3-dfca-49c3-8cf7-b3e059d6d9f8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.420384 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1383/2769] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:10.424854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.436542 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2d20-4b52-4602-8785-f4fce49d9afe admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.442230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.443120 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.443408 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.443687 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] PATH_INFO: `/v3/projects/a2e886d233e04f9185c0712fa652c5aa/users/69ea780305674458a3fa98d35e57b784/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.462678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e886d233e04f9185c0712fa652c5aa, user_id=69ea780305674458a3fa98d35e57b784, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.465358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.477882 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1387/2770] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:10 2022] PUT /identity/v3/projects/a2e886d233e04f9185c0712fa652c5aa/users/69ea780305674458a3fa98d35e57b784/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:10.488412 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.488628 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.488907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.517252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] MFA Rules not processed for user `69ea780305674458a3fa98d35e57b784`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:10.518495 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:10.584843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1384/2771] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:10.590555 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.653681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe45f8a-b1c6-4b84-83cb-346f61f9e606 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.658366 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.659019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.659301 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.659574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.660299 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.662177 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.689392 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1388/2772] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:57:10.693548 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.706028 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a785ea-4f27-4a4c-97b2-29f7148f6ebe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.764274 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.764874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.765158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.765459 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.766189 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.768081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.789055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1385/2773] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:10 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:10.793425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.804240 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad71af-013d-4365-918a-b75571ae9d3a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.808964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.809634 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.809918 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.810194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.810919 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.812550 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.825337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1389/2774] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:10.829984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.840393 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244caa1e-053d-44a8-aede-bb9798060877 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.845161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.846112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.846391 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.846657 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] PATH_INFO: `/v3/projects/175c3a7460054636ae4cf26743dd23f5/users/d39ee406b94e459eb9d4b7d0396daa84/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.863641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175c3a7460054636ae4cf26743dd23f5, user_id=d39ee406b94e459eb9d4b7d0396daa84, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.866986 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.881207 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1386/2775] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:10 2022] PUT /identity/v3/projects/175c3a7460054636ae4cf26743dd23f5/users/d39ee406b94e459eb9d4b7d0396daa84/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:10.885467 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.895458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807810d-2a57-4528-97cb-7e53960a4391 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.955741 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3295f28-a5ef-4090-9f7a-47b4269d0493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.956782 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3295f28-a5ef-4090-9f7a-47b4269d0493 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.957351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3295f28-a5ef-4090-9f7a-47b4269d0493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.957680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3295f28-a5ef-4090-9f7a-47b4269d0493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.958461 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3295f28-a5ef-4090-9f7a-47b4269d0493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.960149 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3295f28-a5ef-4090-9f7a-47b4269d0493 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:10.970913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1390/2776] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:10.975254 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:10.986448 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b7ce6-09d2-40ec-a1aa-eb937f632a14 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:10.990943 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:10.992403 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:10.992705 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:10.992975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] PATH_INFO: `/v3/projects/175c3a7460054636ae4cf26743dd23f5/users/d39ee406b94e459eb9d4b7d0396daa84/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:10.994903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175c3a7460054636ae4cf26743dd23f5, user_id=d39ee406b94e459eb9d4b7d0396daa84, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:10.997574 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:11.008342 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1387/2777] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:10 2022] PUT /identity/v3/projects/175c3a7460054636ae4cf26743dd23f5/users/d39ee406b94e459eb9d4b7d0396daa84/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:11.020230 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263629de-e993-4223-8037-45c7a10d0551 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:11.020442 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263629de-e993-4223-8037-45c7a10d0551 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:11.020748 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263629de-e993-4223-8037-45c7a10d0551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:11.050501 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-263629de-e993-4223-8037-45c7a10d0551 None None] MFA Rules not processed for user `d39ee406b94e459eb9d4b7d0396daa84`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:11.051745 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-263629de-e993-4223-8037-45c7a10d0551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:11.121303 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1391/2778] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:11.132795 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:11.134475 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:11.201580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c19fd-3b28-45b5-b7f6-864a5de0bc53 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:11.205360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca518a7-e3f6-404c-868e-b8930855b2b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:11.206015 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca518a7-e3f6-404c-868e-b8930855b2b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:11.206324 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca518a7-e3f6-404c-868e-b8930855b2b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:11.206608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca518a7-e3f6-404c-868e-b8930855b2b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:11.216465 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca518a7-e3f6-404c-868e-b8930855b2b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:11.217894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca518a7-e3f6-404c-868e-b8930855b2b8 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:11.227693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1388/2779] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:11 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:11.384229 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263629de-e993-4223-8037-45c7a10d0551 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:11.447669 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263629de-e993-4223-8037-45c7a10d0551 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:11.452735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:11.453255 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:11.453539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:11.453992 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] PATH_INFO: `/v3/users/69ea780305674458a3fa98d35e57b784` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:11.455490 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ea780305674458a3fa98d35e57b784)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:11.457580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:11.515254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ea780305674458a3fa98d35e57b784'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:11.528137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ea780305674458a3fa98d35e57b784'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:11.528913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ea780305674458a3fa98d35e57b784'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:11.529520 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ea780305674458a3fa98d35e57b784'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:11.536547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ea780305674458a3fa98d35e57b784'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:11.551153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1392/2780] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:11 2022] DELETE /identity/v3/users/69ea780305674458a3fa98d35e57b784 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:11.557161 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:11.557713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:11.557713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:11.576445 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:11.577650 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:11.637602 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1389/2781] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 12 17:57:11.647476 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:11.649518 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:11.711096 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b490c-410a-4655-8349-52cc07ab9a25 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:11.715094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:11.715750 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:11.716030 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:11.716365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:11.726456 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:11.728139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:11.738690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1393/2782] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:12.024692 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:12.026153 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:12.038726 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b28174-1855-4111-9ecc-4ad457e9aba0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:12.043902 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:12.044824 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.045099 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.045366 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] PATH_INFO: `/v3/projects/a2e886d233e04f9185c0712fa652c5aa` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.046542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e886d233e04f9185c0712fa652c5aa)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:12.048487 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:12.098425 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e886d233e04f9185c0712fa652c5aa', 'request_id': 'req-fbabf69a-7a03-4061-9cbc-b1c1a911932b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.105364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e886d233e04f9185c0712fa652c5aa', 'request_id': 'req-fbabf69a-7a03-4061-9cbc-b1c1a911932b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.110907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e886d233e04f9185c0712fa652c5aa', 'request_id': 'req-fbabf69a-7a03-4061-9cbc-b1c1a911932b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.124075 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] The token cache is being invalidate because project a2e886d233e04f9185c0712fa652c5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:12.124381 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.125154 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.126701 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1390/2783] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:12 2022] DELETE /identity/v3/projects/a2e886d233e04f9185c0712fa652c5aa => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:12.130726 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:12.141660 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcb525-28e0-4f13-bafe-ef9479f1702d service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:12.200893 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:12.201793 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.202077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.202377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] PATH_INFO: `/v3/users/d39ee406b94e459eb9d4b7d0396daa84` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.203644 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39ee406b94e459eb9d4b7d0396daa84)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:12.205732 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:12.255010 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39ee406b94e459eb9d4b7d0396daa84'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.267166 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39ee406b94e459eb9d4b7d0396daa84'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.267882 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39ee406b94e459eb9d4b7d0396daa84'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.268521 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39ee406b94e459eb9d4b7d0396daa84'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.275737 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b9533c40-85c9-46c1-b86a-740149893394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39ee406b94e459eb9d4b7d0396daa84'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.291329 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1394/2784] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:12 2022] DELETE /identity/v3/users/d39ee406b94e459eb9d4b7d0396daa84 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:12.582129 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:12.583619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:12.597980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf69a-7a03-4061-9cbc-b1c1a911932b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:12.660026 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:12.660984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.661307 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.661591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] PATH_INFO: `/v3/projects/175c3a7460054636ae4cf26743dd23f5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.662837 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175c3a7460054636ae4cf26743dd23f5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:12.666008 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:12.716571 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175c3a7460054636ae4cf26743dd23f5', 'request_id': 'req-d606ef93-9bf4-4906-b90f-50a95f936e30'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.722846 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175c3a7460054636ae4cf26743dd23f5', 'request_id': 'req-d606ef93-9bf4-4906-b90f-50a95f936e30'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.729068 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175c3a7460054636ae4cf26743dd23f5', 'request_id': 'req-d606ef93-9bf4-4906-b90f-50a95f936e30'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.741888 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] The token cache is being invalidate because project 175c3a7460054636ae4cf26743dd23f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:12.742184 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.742810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d606ef93-9bf4-4906-b90f-50a95f936e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:12.744483 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1391/2785] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:12 2022] DELETE /identity/v3/projects/175c3a7460054636ae4cf26743dd23f5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:12.749680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.749969 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.750274 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.770074 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:12.771293 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:12.831006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1395/2786] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:12.839718 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.839997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.840298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.859012 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:12.860234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:12.868592 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1392/2787] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:12.872995 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:12.883612 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7d88-d315-4680-b99d-ab03ccc67620 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:12.888094 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:12.888792 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.889070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.889340 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.890093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:57:12.890399 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:12.891999 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:12.904781 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1396/2788] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 12 17:57:12.909450 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:12.919524 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e654f-f1da-48aa-8272-7a67c09de6fb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:12.924372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:12.924995 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:12.925272 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:12.925540 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:12.926229 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:12.928126 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:12.961668 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1393/2789] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:12 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:12.962808 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:12.974694 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ef7f4-6d11-4cc6-9c3a-49987ba93ae6 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:13.036819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:13.037453 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:13.037707 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:13.037990 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:13.039481 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:13.041508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:13.064015 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1397/2790] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:12 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:57:13.068594 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:13.079245 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a0c39-2179-4f2d-a542-c521990c5e8b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:13.084186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:13.084933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:13.085952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:13.086241 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:13.086988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:13.088634 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:13.100152 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1394/2791] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:13.104819 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:13.114816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc452-8e35-42fa-8110-8f7802b64959 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:13.120186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568bdec-c38d-4d4f-8796-4a5f4b9fd575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:13.120857 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bdec-c38d-4d4f-8796-4a5f4b9fd575 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:13.121146 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bdec-c38d-4d4f-8796-4a5f4b9fd575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:13.121462 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568bdec-c38d-4d4f-8796-4a5f4b9fd575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:13.122195 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568bdec-c38d-4d4f-8796-4a5f4b9fd575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:13.124941 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568bdec-c38d-4d4f-8796-4a5f4b9fd575 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:13.135623 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1398/2792] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:13.139793 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:13.150316 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13584c-7357-4c5c-91cc-225ff9f6a481 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:13.154973 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:13.155871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:13.156238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:13.156464 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] PATH_INFO: `/v3/projects/a6a4ef316ff04526ad551531702a16d9/users/93268770d5204652a2025829c738e640/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:13.172215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a4ef316ff04526ad551531702a16d9, user_id=93268770d5204652a2025829c738e640, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:13.174874 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:13.189719 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1395/2793] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:13 2022] PUT /identity/v3/projects/a6a4ef316ff04526ad551531702a16d9/users/93268770d5204652a2025829c738e640/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:13.195625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369f51a-8009-4d8b-b1fb-f539beb8972b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:13.196025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369f51a-8009-4d8b-b1fb-f539beb8972b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:13.196402 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369f51a-8009-4d8b-b1fb-f539beb8972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:13.217134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-9369f51a-8009-4d8b-b1fb-f539beb8972b None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:13.218528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9369f51a-8009-4d8b-b1fb-f539beb8972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:13.279009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1399/2794] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:13.289432 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:13.291220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:13.348779 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaebdb2-bc1b-41b4-8f2c-8bc80d14f3ca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:13.352927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:13.353607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:13.353884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:13.354156 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:13.364222 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:13.365672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:13.375971 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1396/2795] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:14.053213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.053722 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.054597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.091151 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] MFA Rules not processed for user `93268770d5204652a2025829c738e640`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:14.092580 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:14.161231 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1400/2796] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:14.166415 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:14.175905 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e6b30-dc57-4ac8-a867-dfcaf8dfd4ae service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:14.180667 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:14.181364 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.181648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.181924 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.182933 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:14.184964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:14.214273 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1397/2797] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:14 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:14.219594 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:14.231045 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415d198-a841-4927-85e8-132f58eadb1c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:14.291578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:14.292345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.292562 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.292835 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.293530 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:14.295420 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:14.317358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1401/2798] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:14 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:57:14.321736 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:14.332385 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f147a86-ab92-475f-b49e-22c5bf1db55f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:14.337676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:14.338359 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.338637 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.338911 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.339645 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:14.341375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:14.353556 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1398/2799] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:14.358223 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:14.369876 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d0688-bb93-4e88-8cd1-90e7e4fc735e admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:14.375093 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:14.376218 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.376574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.376862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] PATH_INFO: `/v3/projects/cec76a367184446aa6fa325a66d90fe7/users/0aed3a5081e1470889365f41c3fd893d/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.396458 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec76a367184446aa6fa325a66d90fe7, user_id=0aed3a5081e1470889365f41c3fd893d, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:14.400185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:14.413690 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1402/2800] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:14 2022] PUT /identity/v3/projects/cec76a367184446aa6fa325a66d90fe7/users/0aed3a5081e1470889365f41c3fd893d/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:14.418312 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:14.428542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c67285-bb6e-4fdd-bfde-a19064c4d0aa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:14.491525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2269e82-71cf-4162-bedd-e15d32b4bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:14.492232 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2269e82-71cf-4162-bedd-e15d32b4bce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.492517 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2269e82-71cf-4162-bedd-e15d32b4bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.492789 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2269e82-71cf-4162-bedd-e15d32b4bce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.493536 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2269e82-71cf-4162-bedd-e15d32b4bce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:14.495166 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2269e82-71cf-4162-bedd-e15d32b4bce5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:14.507841 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1399/2801] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:14.513092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:14.525486 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbfea6-7bc2-4085-8852-7e39c1627105 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:14.530254 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:14.531139 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:14.531424 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:14.531702 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] PATH_INFO: `/v3/projects/cec76a367184446aa6fa325a66d90fe7/users/0aed3a5081e1470889365f41c3fd893d/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:14.533763 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec76a367184446aa6fa325a66d90fe7, user_id=0aed3a5081e1470889365f41c3fd893d, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:14.536436 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:14.547696 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1403/2802] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:14 2022] PUT /identity/v3/projects/cec76a367184446aa6fa325a66d90fe7/users/0aed3a5081e1470889365f41c3fd893d/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:57:15.236255 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:15.236788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:15.236788 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:15.267004 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] MFA Rules not processed for user `0aed3a5081e1470889365f41c3fd893d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:15.268435 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:15.354171 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1400/2803] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:16.433524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:16.514337 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084907d-ae9e-4e49-bf53-113866e2f0af admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:16.519330 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:16.520242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:16.520523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:16.520797 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] PATH_INFO: `/v3/users/93268770d5204652a2025829c738e640` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:16.522025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93268770d5204652a2025829c738e640)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:16.524527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:16.576587 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93268770d5204652a2025829c738e640'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.591964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93268770d5204652a2025829c738e640'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.592717 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93268770d5204652a2025829c738e640'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.593359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93268770d5204652a2025829c738e640'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.600192 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93268770d5204652a2025829c738e640'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.617869 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1404/2804] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:16 2022] DELETE /identity/v3/users/93268770d5204652a2025829c738e640 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:16.814440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:16.827265 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d63f0-a321-4518-8b77-47593c2c3c10 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:16.891843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:16.892738 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:16.893021 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:16.893301 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] PATH_INFO: `/v3/projects/a6a4ef316ff04526ad551531702a16d9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:16.894511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a4ef316ff04526ad551531702a16d9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:16.896501 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:16.943730 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a4ef316ff04526ad551531702a16d9', 'request_id': 'req-8101c8cb-159e-4033-9fae-5622aa558337'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.953910 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a4ef316ff04526ad551531702a16d9', 'request_id': 'req-8101c8cb-159e-4033-9fae-5622aa558337'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.962985 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a4ef316ff04526ad551531702a16d9', 'request_id': 'req-8101c8cb-159e-4033-9fae-5622aa558337'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.978218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] The token cache is being invalidate because project a6a4ef316ff04526ad551531702a16d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:16.978829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.979137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:16.980798 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1401/2805] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:16 2022] DELETE /identity/v3/projects/a6a4ef316ff04526ad551531702a16d9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:16.984756 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:16.986143 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:16.999203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa2667-4713-42cf-b337-87f16eba654a admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.060621 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.061495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.061777 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.062052 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] PATH_INFO: `/v3/users/0aed3a5081e1470889365f41c3fd893d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.063281 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aed3a5081e1470889365f41c3fd893d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.065263 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.114482 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aed3a5081e1470889365f41c3fd893d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.126699 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aed3a5081e1470889365f41c3fd893d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.127440 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aed3a5081e1470889365f41c3fd893d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.128284 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aed3a5081e1470889365f41c3fd893d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.136086 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-d0941250-92d9-4fc5-a060-5632e801281e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aed3a5081e1470889365f41c3fd893d'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.155459 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1405/2806] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:16 2022] DELETE /identity/v3/users/0aed3a5081e1470889365f41c3fd893d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:17.330529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.331942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:17.344964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101c8cb-159e-4033-9fae-5622aa558337 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.406512 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.407411 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.407693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.407965 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] PATH_INFO: `/v3/projects/cec76a367184446aa6fa325a66d90fe7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.409175 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec76a367184446aa6fa325a66d90fe7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.411106 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.458836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec76a367184446aa6fa325a66d90fe7', 'request_id': 'req-faccb3d3-9559-457f-875d-59eb505ccf7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.467506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec76a367184446aa6fa325a66d90fe7', 'request_id': 'req-faccb3d3-9559-457f-875d-59eb505ccf7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.473234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec76a367184446aa6fa325a66d90fe7', 'request_id': 'req-faccb3d3-9559-457f-875d-59eb505ccf7e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.486887 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] The token cache is being invalidate because project cec76a367184446aa6fa325a66d90fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:17.487252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.487891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-faccb3d3-9559-457f-875d-59eb505ccf7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:17.489998 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1402/2807] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:17 2022] DELETE /identity/v3/projects/cec76a367184446aa6fa325a66d90fe7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:17.496034 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.496576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.496680 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.522178 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:17.524065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:17.588186 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1406/2808] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:17.596401 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.596691 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.597044 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.616050 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:17.617724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:17.627269 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1403/2809] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:17.632365 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.641972 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c0317-89bb-4cdf-a2bc-11ce76d3f974 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.646735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.648037 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.648354 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.648627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.649350 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:57:17.649649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.651225 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.663418 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1407/2810] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:57:17.667810 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.677335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7567-69a5-4b65-b8cf-8308a7b6c70d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.683143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.683955 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.690084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.690084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.690084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.690084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.714688 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1404/2811] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:17 2022] POST /identity/v3/projects => generated 351 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:17.719539 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.730545 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3e209-6d51-4e7d-8019-88ed31e3f3d3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.806090 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.807209 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.807591 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.807888 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.808961 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.812576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.841362 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1408/2812] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:17 2022] POST /identity/v3/users => generated 276 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:57:17.845676 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.859648 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b974f-b606-4ac5-a048-f9cad9836891 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.867931 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.869069 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.869564 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.870051 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.871378 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.874336 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.888000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1405/2813] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:57:17.893097 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.904018 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659149-8f5e-4958-bc53-6f87d41a5605 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.909006 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf3a2c-cb32-4242-ac4b-9571abf3c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.909735 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3a2c-cb32-4242-ac4b-9571abf3c19b admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.910019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3a2c-cb32-4242-ac4b-9571abf3c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.910346 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3a2c-cb32-4242-ac4b-9571abf3c19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.911221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf3a2c-cb32-4242-ac4b-9571abf3c19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.913054 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf3a2c-cb32-4242-ac4b-9571abf3c19b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.924062 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1409/2814] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:17.929351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:17.940494 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d0486-f426-4f19-a910-47aed2672ba4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:17.946228 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:17.947158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.947439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:17.947758 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] PATH_INFO: `/v3/projects/0c0e5f2eee4842b3bd02461f6892f8d3/users/ae1f208025c84bd69463df19c2fdd184/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:17.964079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0e5f2eee4842b3bd02461f6892f8d3, user_id=ae1f208025c84bd69463df19c2fdd184, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:17.967218 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:17.984626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1406/2815] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:17 2022] PUT /identity/v3/projects/0c0e5f2eee4842b3bd02461f6892f8d3/users/ae1f208025c84bd69463df19c2fdd184/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:17.999351 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:17.999647 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.000317 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.027802 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] MFA Rules not processed for user `ae1f208025c84bd69463df19c2fdd184`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:18.029102 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:18.095407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1410/2816] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3218 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:18.099996 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.173707 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c376d-f462-470c-b370-156ae856d227 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.178297 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.178945 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.179226 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.179511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.180204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.182096 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.207853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1407/2817] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:18 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:18.212627 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.224574 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99ee88-a8e9-4283-8a5c-6c5d82ec7ed3 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.282376 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.283058 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.283343 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.283620 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.284348 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.286257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.305922 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1411/2818] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:18 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:18.310192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.321960 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2589451-11cc-4456-8d71-14ecb40217fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.326942 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.327608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.327897 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.328188 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.328903 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.330521 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.342143 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1408/2819] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:18.346353 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.357153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f92955-25c7-453f-ab67-9f979b2a32c0 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.361855 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.362728 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.363009 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.363289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] PATH_INFO: `/v3/projects/079c70e63f1e412bb2fbda528f9adaff/users/d8a8495a9b384999a14e12994e749c1b/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.378872 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] RBAC: Authorizing `identity:create_grant(project_id=079c70e63f1e412bb2fbda528f9adaff, user_id=d8a8495a9b384999a14e12994e749c1b, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.381874 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.396242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1412/2820] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:18 2022] PUT /identity/v3/projects/079c70e63f1e412bb2fbda528f9adaff/users/d8a8495a9b384999a14e12994e749c1b/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:57:18.400539 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.410488 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712849-dd43-4545-9fe4-077168c4a160 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.467397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c5640-bbee-4f92-9fe2-f98433c6d9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.468074 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c5640-bbee-4f92-9fe2-f98433c6d9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.468397 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c5640-bbee-4f92-9fe2-f98433c6d9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.468672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c5640-bbee-4f92-9fe2-f98433c6d9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.469404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c5640-bbee-4f92-9fe2-f98433c6d9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.470997 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c5640-bbee-4f92-9fe2-f98433c6d9e6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.481827 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1409/2821] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:18.486379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.497578 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf261dea-8b89-49e6-9d05-0701a847621b admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.503309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.504210 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.504492 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.504764 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] PATH_INFO: `/v3/projects/079c70e63f1e412bb2fbda528f9adaff/users/d8a8495a9b384999a14e12994e749c1b/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.509158 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079c70e63f1e412bb2fbda528f9adaff, user_id=d8a8495a9b384999a14e12994e749c1b, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.511959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.524377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1413/2822] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:18 2022] PUT /identity/v3/projects/079c70e63f1e412bb2fbda528f9adaff/users/d8a8495a9b384999a14e12994e749c1b/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:18.535436 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.535755 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.536031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.566081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] MFA Rules not processed for user `d8a8495a9b384999a14e12994e749c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:18.567290 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:18.638302 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1410/2823] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:18 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:18.650315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.652803 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:18.712862 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcc06c-7168-45f9-a876-e4fa6339ce94 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.716547 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.717194 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.717511 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.717790 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.727070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.728527 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.738103 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1414/2824] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:18 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:18.814476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:18.825311 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08693cb8-bb6c-4b25-b183-5b40664c8a8f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:18.830711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e64773-7736-4c58-9b56-019889d01ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:18.831640 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e64773-7736-4c58-9b56-019889d01ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:18.831884 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e64773-7736-4c58-9b56-019889d01ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:18.832213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e64773-7736-4c58-9b56-019889d01ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:18.893843 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e64773-7736-4c58-9b56-019889d01ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:18.895421 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e64773-7736-4c58-9b56-019889d01ed2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:18.904778 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1411/2825] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:18 2022] GET /identity/v3/auth/tokens => generated 3218 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:33.949755 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:34.013297 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d3b0-0fcc-4820-86f2-2a4cf31b0c36 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:34.017804 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:34.018667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:34.018945 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:34.019213 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] PATH_INFO: `/v3/users/ae1f208025c84bd69463df19c2fdd184` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:34.020487 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1f208025c84bd69463df19c2fdd184)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:34.023072 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:34.073248 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1f208025c84bd69463df19c2fdd184'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.086829 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1f208025c84bd69463df19c2fdd184'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.087533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1f208025c84bd69463df19c2fdd184'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.088125 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1f208025c84bd69463df19c2fdd184'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.095625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1f208025c84bd69463df19c2fdd184'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.111238 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1415/2826] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:33 2022] DELETE /identity/v3/users/ae1f208025c84bd69463df19c2fdd184 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:34.117952 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:34.118554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:34.118554 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:34.136593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:34.137806 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:34.212964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1412/2827] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:34.223466 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:34.225024 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:34.292211 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7453d30-2784-4c58-8498-1f6fef6ed539 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:34.296246 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:34.296937 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:34.297216 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:34.297495 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:34.313507 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:34.315040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:34.326891 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1416/2828] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 12 17:57:34.625438 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:34.627013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:34.641287 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc33fe-5196-4233-95c4-fd63278070f6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:34.647321 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:34.649335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:34.649335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:34.649335 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] PATH_INFO: `/v3/projects/0c0e5f2eee4842b3bd02461f6892f8d3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:34.650984 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0e5f2eee4842b3bd02461f6892f8d3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:34.652298 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:34.707405 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0e5f2eee4842b3bd02461f6892f8d3', 'request_id': 'req-1d08938c-07f4-4727-a1e7-75496d4f2639'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.720807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0e5f2eee4842b3bd02461f6892f8d3', 'request_id': 'req-1d08938c-07f4-4727-a1e7-75496d4f2639'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.728530 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0e5f2eee4842b3bd02461f6892f8d3', 'request_id': 'req-1d08938c-07f4-4727-a1e7-75496d4f2639'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.743000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] The token cache is being invalidate because project 0c0e5f2eee4842b3bd02461f6892f8d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:34.743000 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.744013 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.745627 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1413/2829] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:34 2022] DELETE /identity/v3/projects/0c0e5f2eee4842b3bd02461f6892f8d3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:57:34.749583 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:34.759799 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a810c-6068-4287-9782-f4cffeb3de90 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:34.826675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:34.826675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:34.826675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:34.826675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] PATH_INFO: `/v3/users/d8a8495a9b384999a14e12994e749c1b` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:34.827233 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a8495a9b384999a14e12994e749c1b)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:34.829339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:34.877622 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a8495a9b384999a14e12994e749c1b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.889766 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a8495a9b384999a14e12994e749c1b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.890446 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a8495a9b384999a14e12994e749c1b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.891180 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a8495a9b384999a14e12994e749c1b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.899132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-8e09c56d-72cf-4d6f-bb54-e739a9b58c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a8495a9b384999a14e12994e749c1b'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:34.915913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1417/2830] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:57:34 2022] DELETE /identity/v3/users/d8a8495a9b384999a14e12994e749c1b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:35.219215 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.220819 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:35.239257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08938c-07f4-4727-a1e7-75496d4f2639 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.301083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.301866 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.302147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.302423 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] PATH_INFO: `/v3/projects/079c70e63f1e412bb2fbda528f9adaff` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.303702 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=079c70e63f1e412bb2fbda528f9adaff)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.305672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.354808 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079c70e63f1e412bb2fbda528f9adaff', 'request_id': 'req-b5aa05b8-6edf-4cf3-b2f2-195651451eae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:35.361559 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079c70e63f1e412bb2fbda528f9adaff', 'request_id': 'req-b5aa05b8-6edf-4cf3-b2f2-195651451eae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:35.367409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079c70e63f1e412bb2fbda528f9adaff', 'request_id': 'req-b5aa05b8-6edf-4cf3-b2f2-195651451eae'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:35.380340 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] The token cache is being invalidate because project 079c70e63f1e412bb2fbda528f9adaff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:57:35.380626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:35.384114 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5aa05b8-6edf-4cf3-b2f2-195651451eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:57:35.386851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1414/2831] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:57:35 2022] DELETE /identity/v3/projects/079c70e63f1e412bb2fbda528f9adaff => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:35.391956 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.392498 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.392625 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.423221 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:35.424544 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:35.485455 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1418/2832] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:35.495964 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.496351 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.496693 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.516410 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:35.517850 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:35.526577 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1415/2833] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:35.531012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.542429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dc62a-4e74-4303-aa10-08758fc2cd97 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.547721 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.548528 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.548859 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.549185 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.550040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:57:35.550404 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.552314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.566203 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1419/2834] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:57:35.571079 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.580818 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c19fb-fbec-432b-af4a-0f1e99c1a74f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.585836 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.586523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.586853 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.587064 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.587786 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.589724 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.618261 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1416/2835] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:35 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:35.622588 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.633506 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088a401-7ad8-499d-abee-fa46e7eeb3fc admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.691579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.692276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.692609 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.693055 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.693771 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.696095 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.716664 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1420/2836] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:35 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:57:35.721137 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.732761 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1e3b-ab41-4bce-9fb8-0660eb89a34f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.737506 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.738167 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.738452 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.738725 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.739456 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.741684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.752916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1417/2837] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:35.760134 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.770359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60f24-2d18-4e97-88d1-17ff634b3b2c admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.775890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e13c-3e87-4043-ada9-2be937c81137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.776573 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e13c-3e87-4043-ada9-2be937c81137 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.776853 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e13c-3e87-4043-ada9-2be937c81137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.777129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e13c-3e87-4043-ada9-2be937c81137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.777839 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0e13c-3e87-4043-ada9-2be937c81137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.779407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0e13c-3e87-4043-ada9-2be937c81137 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.790614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1421/2838] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:35.794975 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:35.805083 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef3dd2-74fb-49be-882d-bb6af337725a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:35.809572 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:35.810434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.810711 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.810981 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] PATH_INFO: `/v3/projects/c1afa869f4de4f4682b4417d041bc7e0/users/9d7f708791ec43e6b48c20b24e7defce/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.825816 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1afa869f4de4f4682b4417d041bc7e0, user_id=9d7f708791ec43e6b48c20b24e7defce, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:35.828460 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:35.842449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1418/2839] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:35 2022] PUT /identity/v3/projects/c1afa869f4de4f4682b4417d041bc7e0/users/9d7f708791ec43e6b48c20b24e7defce/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:35.906333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:35.906579 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:35.906868 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:35.933338 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] MFA Rules not processed for user `9d7f708791ec43e6b48c20b24e7defce`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:35.934538 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:35.998075 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1422/2840] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:36.002619 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.065498 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07933425-e5ae-406f-948a-c3971f4e0efb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.070334 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.070987 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.071257 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.071523 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.072260 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.074164 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.099751 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1419/2841] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:57:36 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:36.104537 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.119733 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef981a-2d0c-4f61-9392-bbe0fc4dfc66 None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.178264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.178964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.179287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.179509 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.180222 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.182232 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.203370 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1423/2842] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:57:36 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:57:36.207332 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.218807 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df0909-9f5f-4b36-8796-ecb58672cee0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.223534 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.224305 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.224593 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.224872 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.226014 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.227716 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.238247 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1420/2843] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:57:36.242705 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.252960 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f111-99c4-499a-89a6-c3ad36a8cfef admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.259025 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.259901 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.260205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.260551 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] PATH_INFO: `/v3/projects/ab7c413262fd4a49b6cdda16bff0ef98/users/b718423803d3487ea86ce3a78d7a6cd3/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.276333 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7c413262fd4a49b6cdda16bff0ef98, user_id=b718423803d3487ea86ce3a78d7a6cd3, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.278959 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.292602 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1424/2844] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:36 2022] PUT /identity/v3/projects/ab7c413262fd4a49b6cdda16bff0ef98/users/b718423803d3487ea86ce3a78d7a6cd3/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:57:36.296871 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.307525 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb9b77-39fb-4f2f-925b-24bbca4282c3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.368339 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894000-2af2-4cff-8f37-c1e6941c7449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.368923 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894000-2af2-4cff-8f37-c1e6941c7449 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.369205 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894000-2af2-4cff-8f37-c1e6941c7449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.369482 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894000-2af2-4cff-8f37-c1e6941c7449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.370239 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894000-2af2-4cff-8f37-c1e6941c7449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.371796 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5894000-2af2-4cff-8f37-c1e6941c7449 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.385641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1421/2845] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:57:36.390092 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.401345 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f02549-ea89-4e82-a8c3-8fe206100169 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.406048 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.406953 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.407259 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.407533 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] PATH_INFO: `/v3/projects/ab7c413262fd4a49b6cdda16bff0ef98/users/b718423803d3487ea86ce3a78d7a6cd3/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.409549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7c413262fd4a49b6cdda16bff0ef98, user_id=b718423803d3487ea86ce3a78d7a6cd3, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.412280 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.423358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1425/2846] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:57:36 2022] PUT /identity/v3/projects/ab7c413262fd4a49b6cdda16bff0ef98/users/b718423803d3487ea86ce3a78d7a6cd3/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:57:36.433946 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.434375 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.434542 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.462360 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] MFA Rules not processed for user `b718423803d3487ea86ce3a78d7a6cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:57:36.463579 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:36.532588 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1422/2847] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:57:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:57:36.544153 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:36.545899 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:57:36.608681 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1e95-b32c-455d-ab8a-e4d0c17528e3 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:36.612405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:36.613053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:36.613339 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:36.613770 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:36.672257 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:36.673686 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:36.682536 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1426/2848] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:57:41.303982 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:57:41.313989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9021bb-180f-430a-ad0a-4d560b9965a1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:57:41.317662 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c86ce-aa96-4ea4-be8a-0abb0695ff40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:57:41.318304 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c86ce-aa96-4ea4-be8a-0abb0695ff40 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:57:41.318622 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c86ce-aa96-4ea4-be8a-0abb0695ff40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:57:41.318916 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c86ce-aa96-4ea4-be8a-0abb0695ff40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:57:41.329158 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c86ce-aa96-4ea4-be8a-0abb0695ff40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:57:41.330671 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c86ce-aa96-4ea4-be8a-0abb0695ff40 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:57:41.341894 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1423/2849] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:57:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:58:04.157524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:04.221662 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2de48a-0a6c-4636-94a8-107ff3ca1bfc service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:04.226113 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:04.226966 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:04.227244 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:04.227586 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] PATH_INFO: `/v3/users/9d7f708791ec43e6b48c20b24e7defce` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:04.229494 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7f708791ec43e6b48c20b24e7defce)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:04.231414 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:04.286947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7f708791ec43e6b48c20b24e7defce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.299249 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7f708791ec43e6b48c20b24e7defce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.299944 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7f708791ec43e6b48c20b24e7defce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.300576 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7f708791ec43e6b48c20b24e7defce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.308112 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7f708791ec43e6b48c20b24e7defce'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.320508 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1427/2850] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:58:04 2022] DELETE /identity/v3/users/9d7f708791ec43e6b48c20b24e7defce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:04.326614 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:04.327204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:04.327204 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:04.346047 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:58:04.347262 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:04.407414 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1424/2851] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:58:04.418133 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:04.419757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:04.481154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3ec06-fa50-4790-925c-29853dea2670 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:04.485137 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:04.485929 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:04.486237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:04.486541 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:04.497522 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:04.499154 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:04.510324 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1428/2852] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:58:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:58:04.795011 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:04.796690 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:04.809032 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ef5a5-32c2-4034-8e40-648643f25596 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:04.814647 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:04.816142 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:04.816600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:04.816898 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] PATH_INFO: `/v3/projects/c1afa869f4de4f4682b4417d041bc7e0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:04.818116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1afa869f4de4f4682b4417d041bc7e0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:04.820007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:04.871386 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1afa869f4de4f4682b4417d041bc7e0', 'request_id': 'req-e12475ca-bcbd-458b-89cf-d8165855bce7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.877511 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1afa869f4de4f4682b4417d041bc7e0', 'request_id': 'req-e12475ca-bcbd-458b-89cf-d8165855bce7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.883088 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1afa869f4de4f4682b4417d041bc7e0', 'request_id': 'req-e12475ca-bcbd-458b-89cf-d8165855bce7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.895516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] The token cache is being invalidate because project c1afa869f4de4f4682b4417d041bc7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:58:04.895804 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.896461 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:04.898007 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1425/2853] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:58:04 2022] DELETE /identity/v3/projects/c1afa869f4de4f4682b4417d041bc7e0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 12 17:58:04.902890 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:04.912630 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd2a2-91ab-42e7-95c3-3100343f1d29 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:04.969513 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:04.970377 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:04.970675 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:04.970923 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] PATH_INFO: `/v3/users/b718423803d3487ea86ce3a78d7a6cd3` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:04.972089 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b718423803d3487ea86ce3a78d7a6cd3)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:04.974013 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.019553 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b718423803d3487ea86ce3a78d7a6cd3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.032049 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b718423803d3487ea86ce3a78d7a6cd3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.032865 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b718423803d3487ea86ce3a78d7a6cd3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.033471 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b718423803d3487ea86ce3a78d7a6cd3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.039943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-2df5ccc5-7ad6-4f9f-a63f-e4756d29899a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b718423803d3487ea86ce3a78d7a6cd3'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.056065 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1429/2854] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:58:04 2022] DELETE /identity/v3/users/b718423803d3487ea86ce3a78d7a6cd3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:05.343762 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.345208 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:05.360829 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12475ca-bcbd-458b-89cf-d8165855bce7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.424891 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.425661 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.425940 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.426213 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] PATH_INFO: `/v3/projects/ab7c413262fd4a49b6cdda16bff0ef98` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.427439 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7c413262fd4a49b6cdda16bff0ef98)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.429504 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.475466 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7c413262fd4a49b6cdda16bff0ef98', 'request_id': 'req-13a4c991-42a9-4aba-b63c-c3135f9d00ce'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.482372 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7c413262fd4a49b6cdda16bff0ef98', 'request_id': 'req-13a4c991-42a9-4aba-b63c-c3135f9d00ce'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.489085 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7c413262fd4a49b6cdda16bff0ef98', 'request_id': 'req-13a4c991-42a9-4aba-b63c-c3135f9d00ce'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.503253 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] The token cache is being invalidate because project ab7c413262fd4a49b6cdda16bff0ef98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:58:05.503537 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.504186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-13a4c991-42a9-4aba-b63c-c3135f9d00ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:05.505735 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1426/2855] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:58:05 2022] DELETE /identity/v3/projects/ab7c413262fd4a49b6cdda16bff0ef98 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:05.511649 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.512140 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.512309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.532264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:58:05.533590 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:05.592104 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1430/2856] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:58:05.599906 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c26242-676e-442d-a617-726a7e661c5f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.600219 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c26242-676e-442d-a617-726a7e661c5f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.600499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c26242-676e-442d-a617-726a7e661c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.620988 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13c26242-676e-442d-a617-726a7e661c5f None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:58:05.622390 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13c26242-676e-442d-a617-726a7e661c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:05.631516 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1427/2857] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:58:05.636107 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.645767 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b5583-364d-443b-9f7c-bee8e39bc80c None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.650315 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.650993 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.651275 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.651549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.652314 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80296) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 12 17:58:05.652614 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.654205 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.666292 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1431/2858] 10.222.0.73 () {58 vars in 1152 bytes} [Thu May 12 17:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 12 17:58:05.670855 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c26242-676e-442d-a617-726a7e661c5f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.680037 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c26242-676e-442d-a617-726a7e661c5f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.685629 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.686371 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.686654 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.686949 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.687766 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.689803 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.716904 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1428/2859] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:58:05 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:58:05.721425 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.732138 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32e8f4-82b8-4aaf-a808-17adc40e54ae admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.789294 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.790610 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.791004 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.791428 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.792532 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.795663 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.819592 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1432/2860] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:58:05 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 12 17:58:05.824146 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.834580 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81470ceb-c421-4566-9a60-f738e01c733e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.840112 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.840794 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.841081 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.841363 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.842093 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.843684 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.856031 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1429/2861] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 12 17:58:05.860276 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.870700 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04201849-dffa-45ff-bb33-4e430fbe364d admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.875469 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49e2eb-8a57-4a1e-b496-a4eb8a00f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.876129 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49e2eb-8a57-4a1e-b496-a4eb8a00f861 admin admin] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.876496 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49e2eb-8a57-4a1e-b496-a4eb8a00f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.876776 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49e2eb-8a57-4a1e-b496-a4eb8a00f861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.877503 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49e2eb-8a57-4a1e-b496-a4eb8a00f861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.879242 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49e2eb-8a57-4a1e-b496-a4eb8a00f861 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.890974 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1433/2862] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:58:05.895220 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:05.907055 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc031761-605e-48b3-bd44-92e015aee511 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:05.912518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:05.913400 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.913678 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.913947 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] PATH_INFO: `/v3/projects/2d73b05719234b3aac9bac52ece2d0b1/users/095859cc3b7e4640a7273893ae01782e/roles/039b03887b8b49dfab13afaf1d54bfc0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.930252 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d73b05719234b3aac9bac52ece2d0b1, user_id=095859cc3b7e4640a7273893ae01782e, role_id=039b03887b8b49dfab13afaf1d54bfc0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:05.932948 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:05.947109 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1430/2863] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:58:05 2022] PUT /identity/v3/projects/2d73b05719234b3aac9bac52ece2d0b1/users/095859cc3b7e4640a7273893ae01782e/roles/039b03887b8b49dfab13afaf1d54bfc0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:05.957711 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:05.958000 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:05.958289 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:05.988822 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.auth.core [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] MFA Rules not processed for user `095859cc3b7e4640a7273893ae01782e`. Rule list: `[]` (Enabled: `True`). {{(pid=80296) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:58:05.990237 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:06.058529 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1434/2864] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:58:06.063869 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.127210 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824e5d1-33dc-4b25-ad8a-5eb1a607377c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.132626 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.133409 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.134752 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.135186 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.136048 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.138103 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.167430 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1431/2865] 10.222.0.73 () {60 vars in 1152 bytes} [Thu May 12 17:58:06 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:58:06.171526 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.182371 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6d4a1-efc4-4718-a873-660d23039e7b None None] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.240405 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.241077 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] REQUEST_METHOD: `POST` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.241358 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.241631 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] PATH_INFO: `/v3/users` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.242309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.244258 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.264449 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1435/2866] 10.222.0.73 () {60 vars in 1143 bytes} [Thu May 12 17:58:06 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 12 17:58:06.268672 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.279600 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba9ae6-5c16-41f2-97ed-58d597c8b152 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.284113 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.284790 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.285102 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.285373 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.286116 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.287686 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.299108 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1432/2867] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:58:06.303373 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.313736 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf3d1d-df8a-4727-9535-9cd941d9597f admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.318805 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.319678 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.319964 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.320272 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] PATH_INFO: `/v3/projects/1ca08839a3a444979efd72769ebced0e/users/f21e057dcbd14c169bf003cdd7c99d52/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.337943 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca08839a3a444979efd72769ebced0e, user_id=f21e057dcbd14c169bf003cdd7c99d52, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.340939 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.354053 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1436/2868] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:58:06 2022] PUT /identity/v3/projects/1ca08839a3a444979efd72769ebced0e/users/f21e057dcbd14c169bf003cdd7c99d52/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:06.358058 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.368199 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4859149-37a1-4697-85e9-bc664980034d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.427192 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ff5e-b9b5-40bc-a435-3426deffae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.427862 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ff5e-b9b5-40bc-a435-3426deffae3d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.428147 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ff5e-b9b5-40bc-a435-3426deffae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.428476 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ff5e-b9b5-40bc-a435-3426deffae3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.429233 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ff5e-b9b5-40bc-a435-3426deffae3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.431434 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ff5e-b9b5-40bc-a435-3426deffae3d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.441963 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1433/2869] 10.222.0.73 () {58 vars in 1121 bytes} [Thu May 12 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 12 17:58:06.446132 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.457524 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fb9f4-7b00-4d0b-95bb-95b179107004 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.462597 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.463472 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.463757 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.464033 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] PATH_INFO: `/v3/projects/1ca08839a3a444979efd72769ebced0e/users/f21e057dcbd14c169bf003cdd7c99d52/roles/9007f8fb775a4cc3875fe578b35f1c1d` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.466040 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca08839a3a444979efd72769ebced0e, user_id=f21e057dcbd14c169bf003cdd7c99d52, role_id=9007f8fb775a4cc3875fe578b35f1c1d)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.468824 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.479359 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1437/2870] 10.222.0.73 () {60 vars in 1482 bytes} [Thu May 12 17:58:06 2022] PUT /identity/v3/projects/1ca08839a3a444979efd72769ebced0e/users/f21e057dcbd14c169bf003cdd7c99d52/roles/9007f8fb775a4cc3875fe578b35f1c1d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:06.490431 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.490865 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.491387 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.520683 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] MFA Rules not processed for user `f21e057dcbd14c169bf003cdd7c99d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:58:06.521927 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:06.592493 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1434/2871] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:58:06.605287 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.607021 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:06.670429 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaa1a0-d02b-44e4-b992-f51d481b5c20 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.675946 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.676619 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.676902 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.677176 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.686854 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.688334 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.698950 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1438/2872] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:58:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:58:06.786468 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:06.797591 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932128a5-a73e-44a1-8c7d-eeb5c49e8207 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:06.801193 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d905ad-c570-42b2-a4e7-105d0e080c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71263b9cdb9d43ca9ce74c3e2069265f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:06.801825 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d905ad-c570-42b2-a4e7-105d0e080c0f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:06.802105 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d905ad-c570-42b2-a4e7-105d0e080c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:06.802374 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d905ad-c570-42b2-a4e7-105d0e080c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:06.860641 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d905ad-c570-42b2-a4e7-105d0e080c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:06.862084 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d905ad-c570-42b2-a4e7-105d0e080c0f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:06.871238 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1435/2873] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:58:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:58:11.637398 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:11.698879 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b706c-4b17-4884-87f7-52452a4a19c2 service cinder] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:11.703569 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:11.704457 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:11.704740 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:11.705012 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] PATH_INFO: `/v3/users/095859cc3b7e4640a7273893ae01782e` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:11.706312 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095859cc3b7e4640a7273893ae01782e)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:11.708264 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:11.755046 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095859cc3b7e4640a7273893ae01782e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:11.766913 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095859cc3b7e4640a7273893ae01782e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:11.767626 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095859cc3b7e4640a7273893ae01782e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:11.768311 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095859cc3b7e4640a7273893ae01782e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:11.775356 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095859cc3b7e4640a7273893ae01782e'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:11.788447 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1439/2874] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:58:11 2022] DELETE /identity/v3/users/095859cc3b7e4640a7273893ae01782e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:11.793907 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:11.794404 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:11.794529 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:11.811980 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] MFA Rules not processed for user `254e338acf294dff9fffc87141257a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 12 17:58:11.813194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:11.872284 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1436/2875] 10.222.0.73 () {58 vars in 957 bytes} [Thu May 12 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 12 17:58:11.882198 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:11.884816 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:11.942118 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7a836-9c09-4ac1-80ff-ecb3eada88c9 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:11.945753 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6076a5f71eed4f8fa47d74a79d6fa709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:11.947379 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:11.947667 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:11.947947 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:11.957215 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:11.958709 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:11.969684 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1440/2876] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 12 17:58:12.239878 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:12.241499 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:12.254967 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d52062-5874-40ad-a220-059984f4b1d8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:12.259882 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:12.260777 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:12.261057 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:12.261326 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] PATH_INFO: `/v3/projects/2d73b05719234b3aac9bac52ece2d0b1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:12.262496 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d73b05719234b3aac9bac52ece2d0b1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:12.264634 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:12.313599 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d73b05719234b3aac9bac52ece2d0b1', 'request_id': 'req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.319399 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d73b05719234b3aac9bac52ece2d0b1', 'request_id': 'req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.324608 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d73b05719234b3aac9bac52ece2d0b1', 'request_id': 'req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.342652 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] The token cache is being invalidate because project 2d73b05719234b3aac9bac52ece2d0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:58:12.343002 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.343713 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.345781 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1437/2877] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:58:12 2022] DELETE /identity/v3/projects/2d73b05719234b3aac9bac52ece2d0b1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:12.349858 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:12.360150 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2537f49-020b-400d-bf54-c1bb2a1d2f48 service neutron] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:12.416549 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:12.417497 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:12.417789 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:12.418042 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] PATH_INFO: `/v3/users/f21e057dcbd14c169bf003cdd7c99d52` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:12.419407 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21e057dcbd14c169bf003cdd7c99d52)` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:12.421202 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:12.468019 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21e057dcbd14c169bf003cdd7c99d52'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.479798 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21e057dcbd14c169bf003cdd7c99d52'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.480475 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21e057dcbd14c169bf003cdd7c99d52'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.481070 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21e057dcbd14c169bf003cdd7c99d52'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.487650 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.notifications [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21e057dcbd14c169bf003cdd7c99d52'} {{(pid=80296) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.500309 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1441/2878] 10.222.0.73 () {58 vars in 1223 bytes} [Thu May 12 17:58:12 2022] DELETE /identity/v3/users/f21e057dcbd14c169bf003cdd7c99d52 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:58:12.770141 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:58:12.771880 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:58:12.788851 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84d2a3-6eef-4f1f-9b90-22fe297c845f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:58:12.850828 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '254e338acf294dff9fffc87141257a5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccf7092b8d0b4d6a9522dc9ba6d91833', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:58:12.851607 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:58:12.851889 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:58:12.852194 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] PATH_INFO: `/v3/projects/1ca08839a3a444979efd72769ebced0e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:58:12.853929 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca08839a3a444979efd72769ebced0e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:58:12.855834 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:58:12.901234 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca08839a3a444979efd72769ebced0e', 'request_id': 'req-fbdc2229-3daa-4623-a650-a8a2552a8c22'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.907449 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca08839a3a444979efd72769ebced0e', 'request_id': 'req-fbdc2229-3daa-4623-a650-a8a2552a8c22'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.912870 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca08839a3a444979efd72769ebced0e', 'request_id': 'req-fbdc2229-3daa-4623-a650-a8a2552a8c22'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.924518 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] The token cache is being invalidate because project 1ca08839a3a444979efd72769ebced0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 12 17:58:12.924800 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.925440 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fbdc2229-3daa-4623-a650-a8a2552a8c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 12 17:58:12.926989 dv-cinder-840525-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1438/2879] 10.222.0.73 () {58 vars in 1232 bytes} [Thu May 12 17:58:12 2022] DELETE /identity/v3/projects/1ca08839a3a444979efd72769ebced0e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 12 17:59:09.133924 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Authenticating user token {{(pid=80296) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 12 17:59:09.135523 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:59:09.228480 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b3dc-211b-48c4-9fe0-72d5013d3bb1 admin admin] Validating token access rules against request {{(pid=80296) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 12 17:59:09.288875 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3704ba4cfa7f4ad8b798636b01ebd95b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed2e282f526442d591d4fa647390748c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80296) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 12 17:59:09.289535 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] REQUEST_METHOD: `GET` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 12 17:59:09.289825 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] SCRIPT_NAME: `/identity` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 12 17:59:09.290100 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80296) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 12 17:59:09.291920 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.fernet_utils [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80296) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 12 17:59:09.303147 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 12 17:59:09.304743 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427b86-94f0-45df-b186-bc59a19d8603 service placement] RBAC: Authorization granted {{(pid=80296) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 12 17:59:09.315115 dv-cinder-840525-2-iscsi devstack@keystone.service[80296]: [pid: 80296|app: 0|req: 1442/2880] 10.222.0.73 () {60 vars in 1365 bytes} [Thu May 12 17:59:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)